-- Logs begin at Wed 2022-09-07 07:14:15 UTC, end at Wed 2022-09-07 08:41:20 UTC. -- Sep 07 07:40:46.939089 dv-cinder-836027-10-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 07 07:40:46.947036 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 7 07:40:46 2022] *** Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: nodename: dv-cinder-836027-10-iscsi Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: machine: x86_64 Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: clock source: unix Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: pcre jit disabled Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: detected number of CPU cores: 4 Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: current working directory: / Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: detected binary path: /usr/bin/uwsgi-core Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: your processes number limit is 47680 Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: your memory page size is 4096 bytes Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: detected max file descriptor number: 2048 Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: lock engine: pthread robust mutexes Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: thunder lock: enabled Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 07 07:40:46.951418 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 07 07:40:46.991267 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: Python main interpreter initialized at 0x555f1a7869c0 Sep 07 07:40:46.991267 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: python threads support enabled Sep 07 07:40:46.991267 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: your server socket listen backlog is limited to 100 connections Sep 07 07:40:46.991267 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: your mercy for graceful operations on workers is 90 seconds Sep 07 07:40:46.991613 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: mapped 403077 bytes (393 KB) for 2 cores Sep 07 07:40:46.991704 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: *** Operational MODE: preforking *** Sep 07 07:40:46.991957 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: *** uWSGI is running in multiple interpreter mode *** Sep 07 07:40:46.991957 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: spawned uWSGI master process (pid: 81465) Sep 07 07:40:46.992184 dv-cinder-836027-10-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 07 07:40:46.993267 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: spawned uWSGI worker 1 (pid: 81475, cores: 1) Sep 07 07:40:46.993267 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: spawned uWSGI worker 2 (pid: 81476, cores: 1) Sep 07 07:40:46.993267 dv-cinder-836027-10-iscsi devstack@keystone.service[81465]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 07 07:40:51.309794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 07:40:51.309794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: from cryptography.utils import int_from_bytes Sep 07 07:40:51.310592 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 07:40:51.310592 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: from cryptography.utils import int_from_bytes Sep 07 07:40:51.326740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 07:40:51.326740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: from cryptography.utils import int_from_bytes Sep 07 07:40:51.327356 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 07:40:51.327356 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: from cryptography.utils import int_from_bytes Sep 07 07:40:51.361790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 07 07:40:51.362022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 07 07:40:51.362151 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] command line args: [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 07 07:40:51.362417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 07 07:40:51.362590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ================================================================================ {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 07 07:40:51.362838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] admin_token = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.363080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] config_dir = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.363265 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.363444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] config_source = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.363620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] debug = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.363814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.365040 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.365040 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.365040 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.365040 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.365040 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.365399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_config_append = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.365399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.365553 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_dir = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.365740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_file = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.365955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.366198 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.366438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.366676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.366851 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.367021 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.367261 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.367504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.367730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.367958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.368208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.368444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.368686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.368923 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.369192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.369670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.369848 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] publish_errors = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.370071 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.370300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.370475 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.370715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.370944 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.371175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] standard_threads = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.371402 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.371639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.371813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.371985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] use_journal = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.372186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] use_json = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.372359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] use_stderr = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.372532 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] use_syslog = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.372759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.373012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.373246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.373500 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.373736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.373980 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.374214 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.374457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.374687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.374928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.375168 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.375411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.375640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.375814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 07 07:40:51.375906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.376018 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 07 07:40:51.376138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.376256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] command line args: [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 07 07:40:51.376340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.376529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 07 07:40:51.376618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.376695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ================================================================================ {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 07 07:40:51.376798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.376950 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] admin_token = **** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.377046 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.377745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] config_dir = [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.377823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.378107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.378192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.378302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.378408 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] config_source = [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.378609 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.378695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] debug = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.378892 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.378992 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.379143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.379222 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.379323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.379402 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.379584 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.379663 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.379816 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.379925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.380028 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.380182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] list_limit = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.380279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] log_config_append = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.380371 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.380520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.380632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.380727 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] log_dir = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.380827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.380970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] log_file = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.381129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.381243 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.381364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.381483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.381595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.381706 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.381824 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.381937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.381937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.382149 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.382260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.382372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.382491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.382610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.382721 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.382839 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.382957 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.383075 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.383196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.383308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.383423 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.383541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.383653 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.383764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.383764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.384401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.384476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.384546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.384615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.384700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.384774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.384774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.384903 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.384981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.385057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] publish_errors = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.385127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.385196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.385264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.385348 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.385348 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.385556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.385556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.385681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.385840 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.385979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.386061 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.386132 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] standard_threads = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.386233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.386312 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.386463 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.386548 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.386642 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.386730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.386807 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] use_journal = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.386908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.386985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] use_json = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.387143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] use_stderr = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.387226 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.387303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] use_syslog = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.387379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.387537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 07:40:51.387615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.387796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.387874 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.388036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.388145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.388298 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.388375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.388527 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.388611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.388760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.388837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.388992 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.389068 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.389255 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.389346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.389515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.389601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.389731 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.389819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.389966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.390039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.390195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.390273 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.390427 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.390510 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.390659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.390737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.390898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.390975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.391130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.391206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.391364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.391441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.391605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.391682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.391838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.391946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.392108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.392214 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.392332 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.392471 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.392570 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.392663 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.393382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.393531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.393631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.393789 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.393912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.394157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.394262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.394382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.394481 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.394622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.394713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.394861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.394946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.395096 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.395173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.395335 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.395448 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.395568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.395679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.395795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.395870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.396019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.396168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.396298 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.396375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.396504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.396574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.396697 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.396809 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.396931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.397052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.397159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.397303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.397381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.397537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.397618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.397778 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.397865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.398028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.398106 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.398262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.398344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.398495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.398582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.398730 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.398808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.398962 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.399044 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.external = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.399202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.399288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.399453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.399531 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.399686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.399763 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.399927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.400014 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.password = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.400185 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.400267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.token = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.400427 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.400508 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.400664 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.400755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.400906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.400984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.401133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.401210 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.401369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.401457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.401608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.401684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.401835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.401915 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.402064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.402151 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.402300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.402376 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.402532 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.402610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.402771 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.402849 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.403005 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.403084 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.403242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.403327 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.403473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.403559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.403713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.403791 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.403959 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.404041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.404215 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.404293 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.404455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.404535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.404694 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.404772 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.404964 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.405040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.405203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.405282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.405445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.405521 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.405689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.405766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.405920 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.405998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.406153 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.406237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.406387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.406490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.406625 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.406705 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.406850 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.406929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.407089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.407167 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.407320 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.407397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.407558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.407637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.407794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.407870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.408032 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.408126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.408299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.408422 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.408609 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.408699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.408842 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.409105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.409195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.409431 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.409509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.409667 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.409744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.409898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.409985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.410142 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.410220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.410369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.410447 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.410600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.410687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.410844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.410932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.411087 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.411164 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.411311 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.411398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.411537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.411615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.411762 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.411847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.411992 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.412098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.412280 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.412430 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.412532 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.412741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.412741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.412941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.413039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.413171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.413273 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.413375 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.413472 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.413576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.413673 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.413775 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.413893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.413989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.414089 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.414193 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.414295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.414397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.414514 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.414619 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] auth.external = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.414750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.414869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.414985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.415090 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.415228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.415333 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.415442 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.415539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] auth.password = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.415682 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.415793 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] auth.token = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.415922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.416032 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.416200 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.416300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.416401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.416502 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.416629 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.416734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.416912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.417037 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.417112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.417182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.417286 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.417371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.417521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.417590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] credential.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.417752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.417835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.418048 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.418127 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.418310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.418409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.418590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.418669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.418870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.418950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.419112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.419264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.419399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.419506 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.419659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.419738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.419901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.420007 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.420216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.420298 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.420479 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.420556 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.420714 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.420795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.420948 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.421026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.421176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.421254 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.421413 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.421492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.421668 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.421746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] federation.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.421898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.421977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.422141 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.422221 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.422382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.422461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.422624 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.422702 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.422856 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.422935 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.423081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.423165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.423314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.423392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.423544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.423624 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.423776 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.423854 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.424134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.424256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.424386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.424492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.424572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.424802 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.424940 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.425075 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.425237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.425318 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.425493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.425573 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.425736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.425815 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.425977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.426058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.426206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.426285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.426438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.426528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.426669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.426748 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.426909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.426987 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.427159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.427249 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.427392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.427469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.427625 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.427707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.427852 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.427938 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.428107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.428220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.428331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.428425 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.428563 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.428654 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.428797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.428876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.429039 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.429117 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.429301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.429393 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.429537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.429640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.429804 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.429883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.430043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.430130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.430287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.430366 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.430520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.430598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.430766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.430853 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.430991 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.431069 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.431233 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.431311 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.431463 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.431549 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.431703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.431780 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.431934 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.432011 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.432190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.432283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.432422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] role.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.432503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.432665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] role.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.432743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.432890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.432977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.433134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.433211 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.433373 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.433451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.433604 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.433691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.433842 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.433921 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.434099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.434178 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.434372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.434458 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.434611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.434688 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.434849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.434927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.435077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.435164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.435318 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.435397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.435553 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.435631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.435786 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.435880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.436025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.436169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.436298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.436522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.436600 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.436757 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.436838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.437006 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.437076 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.437237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.437312 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.437479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.437547 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.437719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.437787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.437950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.438027 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.438195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.438262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.438439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.438506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.438699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.438795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.438938 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.439017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.439194 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.439282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.439422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.439514 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.440215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.440297 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.440614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.440827 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.440909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.441059 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.441136 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.441208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.441302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.441425 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.441534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.441676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.441757 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.441924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.442003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.442174 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.442252 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.442658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.442744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.442907 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.442987 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.443147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.443226 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.443386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.443466 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.443623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.443701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.443857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.443935 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.444126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.444209 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.444357 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.444491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.444593 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.444736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.444817 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.445000 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.445069 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.445237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.445305 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.445487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.445563 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.445729 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.445808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.445967 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.446060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.446252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.446331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.connection = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.446510 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.446682 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.446763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.446988 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.447078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.447227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.447487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.447576 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] resource.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.447894 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.447980 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.448303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.448390 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.448670 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.448755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.448934 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.449092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.449203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.449382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.449469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.449691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.449770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.449931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.450010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.450177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.450266 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.450414 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.450493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] role.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.450661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.450740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] role.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.450895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.450982 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.451139 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.451218 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.451367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.451447 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.451603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.451690 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.451835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.451915 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.451988 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 07 07:40:51.452113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.452367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.452643 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.452944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.453185 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.453449 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.453724 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.453976 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.454227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.454666 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.454907 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.455160 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.456067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.456572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.456918 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.457187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.457451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.457710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.457981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.458220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.458496 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.458734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.459024 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.459284 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.459536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.459802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.460141 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.460370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.460613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] token.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.460854 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.461138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.461396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.461671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.461924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.462166 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.462415 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.462663 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.462916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.463155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.463396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.463635 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.463886 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.464195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.464432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.464678 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.464949 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.465213 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.connection = **** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.465461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.465710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.465952 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.466197 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.466435 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.466674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.466917 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.467156 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.467404 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.467643 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.467891 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.468152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.468441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.468682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.468964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.469199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.469443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.469680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 07:40:51.469851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 07 07:40:51.483261 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.483597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.483897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.499358 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.499588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.499880 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.500998 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.509907 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.536542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.536765 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.537992 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.538228 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.592243 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.596358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.642249 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.642483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.643197 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.643509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.800543 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.800770 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.800862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.800979 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.801065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.801194 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.801279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.801409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.801491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.801622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.801714 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.801896 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.825160 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.825386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.825611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.825695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.825831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.825914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.826045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.826125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.826257 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.826347 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.826469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.826549 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.826699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.826920 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.837501 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.838312 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.851250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.851429 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.851663 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.851894 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.852209 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.852476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.852725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.854575 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.854804 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.855028 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.855247 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.855464 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.855676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.855887 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 07:40:51.870399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.870505 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.870610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 07:40:51.870693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 07:40:51.870956 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.871037 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.871154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 07:40:51.871236 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 07:40:51.871426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.871507 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.871684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.871774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.871940 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.872022 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.872220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.872303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.882343 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.882444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.882621 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.882703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.882890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.882972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.883168 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.883252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.890208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.890300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.893860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.894031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.894148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.894323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.894431 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.894612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.894693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.894900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.894983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.895223 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.895302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.895489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.895576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.895755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.895875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.896022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.896117 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.896314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.896399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.896577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.896658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.896844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.919452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.920239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.920950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.921233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.921756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.922079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.926064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.926332 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.926611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.927263 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.927552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.927834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.932998 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.933285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.934290 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.934579 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.938921 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.939208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.939489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.940310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.940582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.940857 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.941084 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.941169 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.941368 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.941449 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.941656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.941947 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.942233 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.942763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.943035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.943305 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.957581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.957859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.959394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.959677 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.961774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.963544 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.965043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.965337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.966802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.967101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.969596 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.972175 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.973450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.975308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.976703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.977002 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.977281 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.977558 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.977841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.978135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.978535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.978838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.979125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.979411 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.979691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.979970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.994097 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.994381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.994644 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.994904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.996439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:51.996732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.997002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:51.997267 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.001806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.003922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.005009 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.005467 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.005742 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.006568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.008133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.008414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.008496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:52.008696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.008974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.009263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.009655 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.011094 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:52.011396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.011693 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.011979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.012396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.020986 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:52.021265 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.021538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.021812 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.022184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.022470 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.022748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.023970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:52.024285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.024574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.024867 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.025251 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.025528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.025809 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.038983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:52.041919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:52.042428 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:52.044833 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.045426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:52.046352 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.047827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:52.048131 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.048389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.048524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.050875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.052401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:52.052689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.053097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.055676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.057162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:52.059564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.060033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.060449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.060736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.061011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.061091 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:52.064002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.064430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.064753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.065258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.069236 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:52.069534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.069821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.073543 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:52.073837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.074151 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.077593 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:52.078087 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.078586 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.079073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.079577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.080093 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.080641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.081163 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.081665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.081897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 07:40:52.082172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.082258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.082463 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.082692 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.082784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.083028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.083187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.083303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.083584 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.083687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.083876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.084182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.084359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.084462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.084735 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.085007 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.085275 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.085539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 07:40:52.122370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555f1a7869c0 pid: 81476 (default app) Sep 07 07:40:52.125652 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bff1db-d281-476a-97ae-4d9f6f80e049 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:40:52.125942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bff1db-d281-476a-97ae-4d9f6f80e049 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:40:52.126216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bff1db-d281-476a-97ae-4d9f6f80e049 None None] PATH_INFO: `/v3` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:40:52.128819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555f1a7869c0 pid: 81475 (default app) Sep 07 07:40:52.130892 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Wed Sep 7 07:40:52 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:40:58.939006 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3fbf3-7bfb-486c-bbdc-9e396b89dedb None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:40:58.939503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3fbf3-7bfb-486c-bbdc-9e396b89dedb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:40:58.939665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3fbf3-7bfb-486c-bbdc-9e396b89dedb None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:40:58.944822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:40:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:40:58.950068 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52711dc-810b-48dd-9cbd-d220ee370535 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:40:58.950649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52711dc-810b-48dd-9cbd-d220ee370535 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:40:58.950649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52711dc-810b-48dd-9cbd-d220ee370535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:40:59.021143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: INFO dbcounter [None req-f52711dc-810b-48dd-9cbd-d220ee370535 None None] Registered counter for database keystone Sep 07 07:40:59.039829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG oslo_db.sqlalchemy.engines [None req-f52711dc-810b-48dd-9cbd-d220ee370535 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81476) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 07 07:40:59.045973 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writer thread running {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 07 07:40:59.225061 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG passlib.handlers.bcrypt [None req-f52711dc-810b-48dd-9cbd-d220ee370535 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81476) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 07 07:40:59.225359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG passlib.handlers.bcrypt [None req-f52711dc-810b-48dd-9cbd-d220ee370535 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81476) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 07 07:40:59.282642 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f52711dc-810b-48dd-9cbd-d220ee370535 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:40:59.284882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f52711dc-810b-48dd-9cbd-d220ee370535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:40:59.415027 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:40:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 07 07:40:59.919385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3fbf3-7bfb-486c-bbdc-9e396b89dedb None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:40:59.988393 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: INFO dbcounter [None req-02d3fbf3-7bfb-486c-bbdc-9e396b89dedb None None] Registered counter for database keystone Sep 07 07:41:00.009953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG oslo_db.sqlalchemy.engines [None req-02d3fbf3-7bfb-486c-bbdc-9e396b89dedb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81475) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 07 07:41:00.015949 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writer thread running {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 07 07:41:00.134852 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3fbf3-7bfb-486c-bbdc-9e396b89dedb None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:00.139064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf04e9-f41a-4244-81ab-0dcbd89576d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:00.139688 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf04e9-f41a-4244-81ab-0dcbd89576d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:00.139966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf04e9-f41a-4244-81ab-0dcbd89576d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:00.140264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf04e9-f41a-4244-81ab-0dcbd89576d9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:00.149212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf04e9-f41a-4244-81ab-0dcbd89576d9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:00.201410 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: WARNING keystone.common.rbac_enforcer.enforcer [None req-51bf04e9-f41a-4244-81ab-0dcbd89576d9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 07 07:41:00.201684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf04e9-f41a-4244-81ab-0dcbd89576d9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:00.211729 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Wed Sep 7 07:40:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 07:41:00.217638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52711dc-810b-48dd-9cbd-d220ee370535 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:00.232229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52711dc-810b-48dd-9cbd-d220ee370535 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:00.236786 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e2b35-dbf6-4170-b2c8-343d94f894dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:00.238033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e2b35-dbf6-4170-b2c8-343d94f894dc admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:00.238316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e2b35-dbf6-4170-b2c8-343d94f894dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:00.238590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e2b35-dbf6-4170-b2c8-343d94f894dc admin admin] PATH_INFO: `/v3/projects/bb18598c40b6447faef02991e1b9c4ed` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:00.240198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e2b35-dbf6-4170-b2c8-343d94f894dc admin admin] RBAC: Authorizing `identity:get_project(project_id=bb18598c40b6447faef02991e1b9c4ed)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:00.294692 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3d4e2b35-dbf6-4170-b2c8-343d94f894dc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 07 07:41:00.295042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e2b35-dbf6-4170-b2c8-343d94f894dc admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:00.300468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Wed Sep 7 07:41:00 2022] GET /identity/v3/projects/bb18598c40b6447faef02991e1b9c4ed => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:02.448800 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a842a-1ea1-4cbf-9ddd-63ba88bb60b2 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:02.449320 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a842a-1ea1-4cbf-9ddd-63ba88bb60b2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:02.449411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a842a-1ea1-4cbf-9ddd-63ba88bb60b2 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:02.453422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:02.459404 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ff18f-5c73-4a80-b570-88e7aa5dd2f4 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:02.459786 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ff18f-5c73-4a80-b570-88e7aa5dd2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:02.459957 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ff18f-5c73-4a80-b570-88e7aa5dd2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:02.479832 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-327ff18f-5c73-4a80-b570-88e7aa5dd2f4 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:02.481395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-327ff18f-5c73-4a80-b570-88e7aa5dd2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:02.489116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 07 07:41:02.764180 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a842a-1ea1-4cbf-9ddd-63ba88bb60b2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:02.776153 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a842a-1ea1-4cbf-9ddd-63ba88bb60b2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:02.780304 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c76100-a392-4aa0-a596-7006ba0459b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:02.780896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c76100-a392-4aa0-a596-7006ba0459b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:02.781175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c76100-a392-4aa0-a596-7006ba0459b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:02.781448 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c76100-a392-4aa0-a596-7006ba0459b7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:02.789567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c76100-a392-4aa0-a596-7006ba0459b7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:02.791311 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c76100-a392-4aa0-a596-7006ba0459b7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:02.800613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Wed Sep 7 07:41:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 07:41:02.805832 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ff18f-5c73-4a80-b570-88e7aa5dd2f4 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:02.814857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ff18f-5c73-4a80-b570-88e7aa5dd2f4 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:02.819417 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa89bf0-2f50-41af-8b93-2eac33a1802c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:02.820035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa89bf0-2f50-41af-8b93-2eac33a1802c admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:02.820340 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa89bf0-2f50-41af-8b93-2eac33a1802c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:02.820609 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa89bf0-2f50-41af-8b93-2eac33a1802c admin admin] PATH_INFO: `/v3/users/984123d0eaef4929af01f39a46ebd51e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:02.821892 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa89bf0-2f50-41af-8b93-2eac33a1802c admin admin] RBAC: Authorizing `identity:get_user(user_id=984123d0eaef4929af01f39a46ebd51e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:02.823731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa89bf0-2f50-41af-8b93-2eac33a1802c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:02.825272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Wed Sep 7 07:41:02 2022] GET /identity/v3/users/984123d0eaef4929af01f39a46ebd51e => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:05.049343 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc49a0-d805-4428-9343-66b1110c81e2 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:05.049855 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc49a0-d805-4428-9343-66b1110c81e2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:05.049855 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc49a0-d805-4428-9343-66b1110c81e2 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:05.051177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:41:05.056494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fe8eb-855a-45a6-a23f-568467f496c5 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:05.056883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fe8eb-855a-45a6-a23f-568467f496c5 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:05.057105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fe8eb-855a-45a6-a23f-568467f496c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:05.075647 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b34fe8eb-855a-45a6-a23f-568467f496c5 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:05.076838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b34fe8eb-855a-45a6-a23f-568467f496c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:05.121506 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0728e-5fd2-47a7-b249-d681e25e69a8 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:05.121788 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0728e-5fd2-47a7-b249-d681e25e69a8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:05.122068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0728e-5fd2-47a7-b249-d681e25e69a8 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:05.123571 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:41:05.129580 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cf57f-3efd-4355-8cad-176dbe45d030 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:05.129912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cf57f-3efd-4355-8cad-176dbe45d030 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:05.130189 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cf57f-3efd-4355-8cad-176dbe45d030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:05.157033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:05.173368 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG passlib.handlers.bcrypt [None req-886cf57f-3efd-4355-8cad-176dbe45d030 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81475) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 07 07:41:05.173763 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG passlib.handlers.bcrypt [None req-886cf57f-3efd-4355-8cad-176dbe45d030 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81475) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 07 07:41:05.223382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-886cf57f-3efd-4355-8cad-176dbe45d030 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:05.224975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-886cf57f-3efd-4355-8cad-176dbe45d030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:05.325501 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:05.430169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fe8eb-855a-45a6-a23f-568467f496c5 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:05.456350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:05.536029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fe8eb-855a-45a6-a23f-568467f496c5 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:05.570360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afc76c-6228-4710-a415-67610cbea0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:05.570995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afc76c-6228-4710-a415-67610cbea0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:05.571279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afc76c-6228-4710-a415-67610cbea0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:05.571554 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afc76c-6228-4710-a415-67610cbea0f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:05.582626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44afc76c-6228-4710-a415-67610cbea0f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:05.591370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-44afc76c-6228-4710-a415-67610cbea0f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:05.591370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:05.591370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:05.591370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:05.591370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:05.591370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:05.591370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:05.591370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:05.591370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:05.591370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:05.591370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:05.591370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:05.591370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:05.591370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:05.591370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:05.591370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:05.592394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:05.592394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:05.592394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:05.592394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:05.592394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:05.592394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:05.592394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:05.592394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:05.592394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:05.592394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:05.592394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:05.592394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:05.592394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:05.592394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:05.592394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:05.592394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:05.593250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:05.593250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:05.593250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:05.593250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:05.593250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:05.593250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:05.593250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:05.593250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:05.593250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:05.593250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:05.593250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:05.593250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:05.593250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:41:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:05.595390 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cf57f-3efd-4355-8cad-176dbe45d030 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:05.598870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afc76c-6228-4710-a415-67610cbea0f4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:05.681308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:05.697301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cf57f-3efd-4355-8cad-176dbe45d030 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:05.699614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:05.700494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afc76c-6228-4710-a415-67610cbea0f4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:05.735578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7258c9-e8f9-47db-8f6c-7b49816d2e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:05.736479 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7258c9-e8f9-47db-8f6c-7b49816d2e38 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:05.737206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03ee0a-efa9-4672-9d0b-c53f69b96cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:05.737322 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7258c9-e8f9-47db-8f6c-7b49816d2e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:05.737763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03ee0a-efa9-4672-9d0b-c53f69b96cdb None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:05.737867 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7258c9-e8f9-47db-8f6c-7b49816d2e38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:05.738081 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03ee0a-efa9-4672-9d0b-c53f69b96cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:05.738396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03ee0a-efa9-4672-9d0b-c53f69b96cdb None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:05.739140 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03ee0a-efa9-4672-9d0b-c53f69b96cdb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:05.739484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03ee0a-efa9-4672-9d0b-c53f69b96cdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:05.741104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03ee0a-efa9-4672-9d0b-c53f69b96cdb None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:05.753412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:05.759155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03ee0a-efa9-4672-9d0b-c53f69b96cdb None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:05.807224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7258c9-e8f9-47db-8f6c-7b49816d2e38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:05.808778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7258c9-e8f9-47db-8f6c-7b49816d2e38 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:05.866502 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03ee0a-efa9-4672-9d0b-c53f69b96cdb None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:05.900915 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36b4c0-96aa-4af7-94e4-6c9ba7e77843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:05.901546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36b4c0-96aa-4af7-94e4-6c9ba7e77843 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:05.901825 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36b4c0-96aa-4af7-94e4-6c9ba7e77843 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:05.902102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36b4c0-96aa-4af7-94e4-6c9ba7e77843 None admin] PATH_INFO: `/v3/users/984123d0eaef4929af01f39a46ebd51e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:05.903230 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36b4c0-96aa-4af7-94e4-6c9ba7e77843 None admin] RBAC: Authorizing `identity:get_user(user_id=984123d0eaef4929af01f39a46ebd51e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:05.904721 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36b4c0-96aa-4af7-94e4-6c9ba7e77843 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:05.905379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 8/16] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 07:41:05.906281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:05 2022] GET /identity/v3/users/984123d0eaef4929af01f39a46ebd51e => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:05.911011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7258c9-e8f9-47db-8f6c-7b49816d2e38 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:05.911707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36b4c0-96aa-4af7-94e4-6c9ba7e77843 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:05.913872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:05.928698 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:06.011344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7258c9-e8f9-47db-8f6c-7b49816d2e38 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:06.014042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36b4c0-96aa-4af7-94e4-6c9ba7e77843 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:06.044187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db912845-1911-4444-9977-a6824438d0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:06.045026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db912845-1911-4444-9977-a6824438d0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:06.045300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db912845-1911-4444-9977-a6824438d0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:06.045568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db912845-1911-4444-9977-a6824438d0e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:06.046894 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f80d72-532c-434e-b903-72ec5bbce8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:06.047518 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f80d72-532c-434e-b903-72ec5bbce8cc None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:06.047794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f80d72-532c-434e-b903-72ec5bbce8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:06.048066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f80d72-532c-434e-b903-72ec5bbce8cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:06.049712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f80d72-532c-434e-b903-72ec5bbce8cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:06.051154 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f80d72-532c-434e-b903-72ec5bbce8cc None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:06.052755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 10/18] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:06.058525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f80d72-532c-434e-b903-72ec5bbce8cc None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:06.064473 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db912845-1911-4444-9977-a6824438d0e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:06.066535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db912845-1911-4444-9977-a6824438d0e0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:06.085186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-db912845-1911-4444-9977-a6824438d0e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:41:06.085186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:06.085186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:06.085186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:06.085186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:06.085186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:06.085186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:06.085186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:06.085186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:06.085186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:06.085186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:06.085186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:06.085186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:41:06.085186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:41:06.085186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:41:06.085186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:06.086073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:41:06.086878 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:41:06.086878 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:41:06.086878 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:06.086878 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 9/19] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:06.093139 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db912845-1911-4444-9977-a6824438d0e0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:06.142506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:06.148505 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:06.162781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f80d72-532c-434e-b903-72ec5bbce8cc None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:06.193686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db912845-1911-4444-9977-a6824438d0e0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:06.197636 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f65b5-d0ae-4670-b577-9cdd78340846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:06.198241 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f65b5-d0ae-4670-b577-9cdd78340846 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:06.198520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f65b5-d0ae-4670-b577-9cdd78340846 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:06.198792 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f65b5-d0ae-4670-b577-9cdd78340846 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:06.199511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f65b5-d0ae-4670-b577-9cdd78340846 None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.domain.id=default, user.id=984123d0eaef4929af01f39a46ebd51e) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:06.199814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f65b5-d0ae-4670-b577-9cdd78340846 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:06.201284 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f65b5-d0ae-4670-b577-9cdd78340846 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:06.207892 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 11/20] 10.222.0.26 () {56 vars in 1322 bytes} [Wed Sep 7 07:41:06 2022] GET /identity/v3/role_assignments?user.id=984123d0eaef4929af01f39a46ebd51e&scope.domain.id=default&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:06.226941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973e9ce-22bb-4eb5-bb1d-24e417291d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:06.227594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973e9ce-22bb-4eb5-bb1d-24e417291d36 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:06.227868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973e9ce-22bb-4eb5-bb1d-24e417291d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:06.228167 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973e9ce-22bb-4eb5-bb1d-24e417291d36 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:06.228822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973e9ce-22bb-4eb5-bb1d-24e417291d36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:06.229120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973e9ce-22bb-4eb5-bb1d-24e417291d36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:06.230529 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973e9ce-22bb-4eb5-bb1d-24e417291d36 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:06.250676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 10/21] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:08.640881 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d414d1-9a62-41c0-8f03-b566e40ce908 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:08.641352 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d414d1-9a62-41c0-8f03-b566e40ce908 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:08.641352 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d414d1-9a62-41c0-8f03-b566e40ce908 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:08.643591 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 12/22] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:08.649703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9ef44-ab65-4563-bfcb-ce61263552e6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:08.650570 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9ef44-ab65-4563-bfcb-ce61263552e6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:08.651155 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9ef44-ab65-4563-bfcb-ce61263552e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:08.672553 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a3b9ef44-ab65-4563-bfcb-ce61263552e6 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:08.674546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a3b9ef44-ab65-4563-bfcb-ce61263552e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:08.769413 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 11/23] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:09.066226 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf505306-b048-4f32-8b68-e41e5b06408e None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:09.066927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf505306-b048-4f32-8b68-e41e5b06408e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:09.067429 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf505306-b048-4f32-8b68-e41e5b06408e None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:09.069363 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 13/24] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:09.075209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965982e-37f4-4b30-85be-2930b0a2be05 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:09.075776 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965982e-37f4-4b30-85be-2930b0a2be05 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:09.076312 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965982e-37f4-4b30-85be-2930b0a2be05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:09.101721 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0965982e-37f4-4b30-85be-2930b0a2be05 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:09.103350 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0965982e-37f4-4b30-85be-2930b0a2be05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:09.159642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:09.190900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 12/25] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:09.310286 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c66d13-efab-4d4a-93fc-51f6ffdb1a83 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:09.310965 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c66d13-efab-4d4a-93fc-51f6ffdb1a83 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:09.311451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c66d13-efab-4d4a-93fc-51f6ffdb1a83 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:09.313558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 14/26] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:09.315637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965982e-37f4-4b30-85be-2930b0a2be05 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:09.320818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84603d20-6354-4cee-bbcf-501890e4ed75 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:09.321419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84603d20-6354-4cee-bbcf-501890e4ed75 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:09.321908 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84603d20-6354-4cee-bbcf-501890e4ed75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:09.349774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-84603d20-6354-4cee-bbcf-501890e4ed75 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:09.351324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-84603d20-6354-4cee-bbcf-501890e4ed75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:09.517013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965982e-37f4-4b30-85be-2930b0a2be05 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:09.522079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 15/27] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:09.546790 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ef6c3-3cbb-44a2-a2a1-3bd314085726 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:09.547233 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ef6c3-3cbb-44a2-a2a1-3bd314085726 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:09.547624 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ef6c3-3cbb-44a2-a2a1-3bd314085726 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:09.549501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 16/28] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:09.554608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47455d1-785a-4445-ab92-919d34407720 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:09.555030 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47455d1-785a-4445-ab92-919d34407720 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:09.555415 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47455d1-785a-4445-ab92-919d34407720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:09.588589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e47455d1-785a-4445-ab92-919d34407720 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:09.590916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e47455d1-785a-4445-ab92-919d34407720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:09.591710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc3440-ebec-4c35-ad88-ae28eee6ebb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:09.592572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc3440-ebec-4c35-ad88-ae28eee6ebb8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:09.593049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc3440-ebec-4c35-ad88-ae28eee6ebb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:09.593463 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc3440-ebec-4c35-ad88-ae28eee6ebb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:09.618246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc3440-ebec-4c35-ad88-ae28eee6ebb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:09.632221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-fecc3440-ebec-4c35-ad88-ae28eee6ebb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:09.632221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:09.632221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:09.632221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:09.632221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:09.632221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:09.632221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:09.632221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:09.632221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:09.632221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:09.632221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:09.632221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:09.632221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:09.632221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:09.632221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:09.632221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:09.633154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:09.633154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:09.633154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:09.633154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:09.633154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:09.633154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:09.633154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:09.633154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:09.633154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:09.633154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:09.633154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:09.633154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:09.633154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:09.633154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:09.633154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:09.633154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:09.633985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:09.633985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:09.633985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:09.633985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:09.633985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:09.633985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:09.633985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:09.633985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:09.633985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:09.633985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:09.633985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:09.633985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:09.639966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 13/29] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:41:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:09.641031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc3440-ebec-4c35-ad88-ae28eee6ebb8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:09.719289 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:09.723266 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 17/30] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:09.724255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47455d1-785a-4445-ab92-919d34407720 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:09.731171 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:09.831200 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc3440-ebec-4c35-ad88-ae28eee6ebb8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:09.866942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96952d6-82f5-4acb-8131-8faee168cfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:09.867496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96952d6-82f5-4acb-8131-8faee168cfdc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:09.867773 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96952d6-82f5-4acb-8131-8faee168cfdc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:09.868041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96952d6-82f5-4acb-8131-8faee168cfdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:09.878536 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96952d6-82f5-4acb-8131-8faee168cfdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:09.880173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96952d6-82f5-4acb-8131-8faee168cfdc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:09.897819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-a96952d6-82f5-4acb-8131-8faee168cfdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:41:09.897819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:09.897819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:09.897819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:09.897819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:09.897819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:09.897819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:09.897819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:09.897819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:09.897819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:09.897819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:09.897819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:09.897819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:41:09.897819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:41:09.897819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:41:09.897819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:09.898719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:41:09.899499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:41:09.899499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:41:09.899499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:09.899499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 14/31] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:09.899703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96952d6-82f5-4acb-8131-8faee168cfdc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:09.968134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47455d1-785a-4445-ab92-919d34407720 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:10.000448 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:10.010564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96952d6-82f5-4acb-8131-8faee168cfdc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:10.045497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c3b26-255b-4efd-bac6-78a795316d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:10.046155 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c3b26-255b-4efd-bac6-78a795316d71 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:10.046444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c3b26-255b-4efd-bac6-78a795316d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:10.046722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c3b26-255b-4efd-bac6-78a795316d71 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:10.047395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c3b26-255b-4efd-bac6-78a795316d71 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:10.048816 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c3b26-255b-4efd-bac6-78a795316d71 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:10.056819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3bdac-d660-4c31-8a58-64aac7485cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:10.057443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3bdac-d660-4c31-8a58-64aac7485cca None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:10.057727 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3bdac-d660-4c31-8a58-64aac7485cca None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:10.058015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3bdac-d660-4c31-8a58-64aac7485cca None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:10.058742 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3bdac-d660-4c31-8a58-64aac7485cca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:10.059066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3bdac-d660-4c31-8a58-64aac7485cca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:10.062110 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3bdac-d660-4c31-8a58-64aac7485cca None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:10.075083 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 18/32] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:10.076306 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3bdac-d660-4c31-8a58-64aac7485cca None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:10.155716 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:10.195321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3bdac-d660-4c31-8a58-64aac7485cca None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:10.196839 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 15/33] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Sep 7 07:41:09 2022] POST /identity/v3/roles => generated 225 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:41:10.197904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c3b26-255b-4efd-bac6-78a795316d71 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:10.268032 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c588c-0ec8-4149-bbde-b56e3b171eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:10.269179 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c588c-0ec8-4149-bbde-b56e3b171eff None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:10.269467 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c588c-0ec8-4149-bbde-b56e3b171eff None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:10.269746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c588c-0ec8-4149-bbde-b56e3b171eff None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:10.270441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c588c-0ec8-4149-bbde-b56e3b171eff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:10.270760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c588c-0ec8-4149-bbde-b56e3b171eff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:10.272294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c588c-0ec8-4149-bbde-b56e3b171eff None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:10.314293 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 19/34] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:10.315585 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c588c-0ec8-4149-bbde-b56e3b171eff None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:10.405248 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c3b26-255b-4efd-bac6-78a795316d71 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:10.459817 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d21fc7-ac5a-4e01-8e1c-8bdcd455f99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:10.460449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d21fc7-ac5a-4e01-8e1c-8bdcd455f99e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:10.460746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d21fc7-ac5a-4e01-8e1c-8bdcd455f99e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:10.461023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d21fc7-ac5a-4e01-8e1c-8bdcd455f99e None admin] PATH_INFO: `/v3/users/984123d0eaef4929af01f39a46ebd51e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:10.462793 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d21fc7-ac5a-4e01-8e1c-8bdcd455f99e None admin] RBAC: Authorizing `identity:get_user(user_id=984123d0eaef4929af01f39a46ebd51e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:10.464392 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d21fc7-ac5a-4e01-8e1c-8bdcd455f99e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:10.469752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 16/35] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:10 2022] GET /identity/v3/users/984123d0eaef4929af01f39a46ebd51e => generated 247 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:10.469752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d21fc7-ac5a-4e01-8e1c-8bdcd455f99e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:10.479094 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c588c-0ec8-4149-bbde-b56e3b171eff None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:10.491098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:10.520241 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:10.531323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe88729-a561-4c0f-8b8b-29e9338417ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:10.531944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe88729-a561-4c0f-8b8b-29e9338417ee None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:10.532401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe88729-a561-4c0f-8b8b-29e9338417ee None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:10.532551 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe88729-a561-4c0f-8b8b-29e9338417ee None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:10.533266 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe88729-a561-4c0f-8b8b-29e9338417ee None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:10.535218 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe88729-a561-4c0f-8b8b-29e9338417ee None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:10.596529 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d21fc7-ac5a-4e01-8e1c-8bdcd455f99e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:10.632034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f94fba-5d0f-4f5e-bdae-829591313ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:10.632677 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f94fba-5d0f-4f5e-bdae-829591313ce5 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:10.632963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f94fba-5d0f-4f5e-bdae-829591313ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:10.633240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f94fba-5d0f-4f5e-bdae-829591313ce5 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:10.633894 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f94fba-5d0f-4f5e-bdae-829591313ce5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:10.635333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f94fba-5d0f-4f5e-bdae-829591313ce5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:10.655874 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 20/36] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Sep 7 07:41:10 2022] POST /identity/v3/roles => generated 219 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:41:10.657539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067fcde-fd77-4610-b779-853ae864ab41 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:10.657940 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067fcde-fd77-4610-b779-853ae864ab41 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:10.658174 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067fcde-fd77-4610-b779-853ae864ab41 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:10.659414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 21/37] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:10.660264 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067fcde-fd77-4610-b779-853ae864ab41 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:10.668192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 17/38] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 7 07:41:10 2022] POST /identity/v3/projects => generated 301 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:41:10.670100 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ef2c0-e9cf-467f-aa35-444545f2dbbf None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:10.670805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ef2c0-e9cf-467f-aa35-444545f2dbbf None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:10.671305 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ef2c0-e9cf-467f-aa35-444545f2dbbf None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:10.673067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 18/39] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:10.681804 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02069494-b6dc-4a3b-a207-1c1aafcc265f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:10.682444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02069494-b6dc-4a3b-a207-1c1aafcc265f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:10.682928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02069494-b6dc-4a3b-a207-1c1aafcc265f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:10.708426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-02069494-b6dc-4a3b-a207-1c1aafcc265f None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:10.711121 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-02069494-b6dc-4a3b-a207-1c1aafcc265f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:10.801827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:10.807273 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067fcde-fd77-4610-b779-853ae864ab41 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:10.816142 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 19/40] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:10.816142 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609f536-98e8-46ad-8c04-6d8eb213fbe4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:10.816142 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609f536-98e8-46ad-8c04-6d8eb213fbe4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:10.816142 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609f536-98e8-46ad-8c04-6d8eb213fbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:10.851173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b609f536-98e8-46ad-8c04-6d8eb213fbe4 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:10.852433 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b609f536-98e8-46ad-8c04-6d8eb213fbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:10.875102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305da045-4042-46f5-8990-b445e3240fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:10.876179 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305da045-4042-46f5-8990-b445e3240fbf None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:10.876685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305da045-4042-46f5-8990-b445e3240fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:10.877212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305da045-4042-46f5-8990-b445e3240fbf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:10.881377 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305da045-4042-46f5-8990-b445e3240fbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:10.884202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305da045-4042-46f5-8990-b445e3240fbf None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:10.888235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 22/41] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:10.894725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305da045-4042-46f5-8990-b445e3240fbf None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:10.941564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 20/42] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:10.981039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:11.030707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305da045-4042-46f5-8990-b445e3240fbf None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:11.066099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60201f-6c10-4ad5-91f8-eba0e1448ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:11.066758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60201f-6c10-4ad5-91f8-eba0e1448ad0 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:11.067039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60201f-6c10-4ad5-91f8-eba0e1448ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:11.067327 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60201f-6c10-4ad5-91f8-eba0e1448ad0 None admin] PATH_INFO: `/v3/domains/default/users/984123d0eaef4929af01f39a46ebd51e/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:11.069788 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60201f-6c10-4ad5-91f8-eba0e1448ad0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=984123d0eaef4929af01f39a46ebd51e, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:11.071299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60201f-6c10-4ad5-91f8-eba0e1448ad0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:11.092124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 23/43] 10.222.0.26 () {58 vars in 1361 bytes} [Wed Sep 7 07:41:10 2022] PUT /identity/v3/domains/default/users/984123d0eaef4929af01f39a46ebd51e/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:41:11.117210 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609f536-98e8-46ad-8c04-6d8eb213fbe4 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:11.223101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60201f-6c10-4ad5-91f8-eba0e1448ad0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:11.235485 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609f536-98e8-46ad-8c04-6d8eb213fbe4 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:11.268734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:11.277599 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539e1f4-2270-4e0a-a34e-3957808b39ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:11.278239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539e1f4-2270-4e0a-a34e-3957808b39ef None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:11.278534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539e1f4-2270-4e0a-a34e-3957808b39ef None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:11.278782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539e1f4-2270-4e0a-a34e-3957808b39ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:11.280224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539e1f4-2270-4e0a-a34e-3957808b39ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:11.282030 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539e1f4-2270-4e0a-a34e-3957808b39ef None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:11.284148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 21/44] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:11.289613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539e1f4-2270-4e0a-a34e-3957808b39ef None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:11.344458 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60201f-6c10-4ad5-91f8-eba0e1448ad0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:11.360232 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:11.389394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd85ce-9995-4d4a-b767-24e2039cec2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:11.390005 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd85ce-9995-4d4a-b767-24e2039cec2d None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:11.390282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd85ce-9995-4d4a-b767-24e2039cec2d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:11.390552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd85ce-9995-4d4a-b767-24e2039cec2d None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:11.391225 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd85ce-9995-4d4a-b767-24e2039cec2d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:11.393261 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd85ce-9995-4d4a-b767-24e2039cec2d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:11.395655 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539e1f4-2270-4e0a-a34e-3957808b39ef None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:11.410057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 24/45] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Sep 7 07:41:11 2022] POST /identity/v3/roles => generated 223 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:41:11.440184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73923f0-f269-430b-b820-d384c25202aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:11.440995 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73923f0-f269-430b-b820-d384c25202aa None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:11.441340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73923f0-f269-430b-b820-d384c25202aa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:11.441675 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73923f0-f269-430b-b820-d384c25202aa None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:11.442412 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73923f0-f269-430b-b820-d384c25202aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:11.444275 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73923f0-f269-430b-b820-d384c25202aa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:11.477238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 22/46] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 7 07:41:11 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:41:13.253209 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28d16e-a0f8-4e2b-b348-afc5b16be69c None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:13.253728 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28d16e-a0f8-4e2b-b348-afc5b16be69c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:13.253900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28d16e-a0f8-4e2b-b348-afc5b16be69c None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:13.255115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 25/47] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:13.260414 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fed36e-bd52-417e-ba25-8dbc13099f0e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:13.260813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fed36e-bd52-417e-ba25-8dbc13099f0e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:13.260978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fed36e-bd52-417e-ba25-8dbc13099f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:13.279034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-75fed36e-bd52-417e-ba25-8dbc13099f0e None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:13.280240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-75fed36e-bd52-417e-ba25-8dbc13099f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:13.323918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:13.347499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 23/48] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:13.618928 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28d16e-a0f8-4e2b-b348-afc5b16be69c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:13.662552 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd3242-2a40-4a8f-a67e-56acd71d8b2f None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:13.662832 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd3242-2a40-4a8f-a67e-56acd71d8b2f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:13.663110 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd3242-2a40-4a8f-a67e-56acd71d8b2f None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:13.664533 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 24/49] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:41:13.670506 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69907ba9-a423-4463-9462-feb55104b6e6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:13.670792 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69907ba9-a423-4463-9462-feb55104b6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:13.671125 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69907ba9-a423-4463-9462-feb55104b6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:13.692347 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-69907ba9-a423-4463-9462-feb55104b6e6 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:13.693816 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-69907ba9-a423-4463-9462-feb55104b6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:13.834349 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28d16e-a0f8-4e2b-b348-afc5b16be69c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:13.867042 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 25/50] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:13.869128 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966afc1-ebbd-4a3f-a16b-4ae1ea468be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:13.869797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966afc1-ebbd-4a3f-a16b-4ae1ea468be3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:13.870143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966afc1-ebbd-4a3f-a16b-4ae1ea468be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:13.870481 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966afc1-ebbd-4a3f-a16b-4ae1ea468be3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:13.880961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966afc1-ebbd-4a3f-a16b-4ae1ea468be3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:13.885860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-c966afc1-ebbd-4a3f-a16b-4ae1ea468be3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:13.885860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:13.885860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:13.885860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:13.885860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:13.885860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:13.885860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:13.885860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:13.885860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:13.885860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:13.885860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:13.885860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:13.885860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:13.885860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:13.885860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:13.885860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:13.886661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:13.886661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:13.886661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:13.886661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:13.886661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:13.886661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:13.886661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:13.886661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:13.886661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:13.886661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:13.886661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:13.886661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:13.886661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:13.886661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:13.886661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:13.886661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:13.887398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:13.887398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:13.887398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:13.887398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:13.887398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:13.887398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:13.887398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:13.887398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:13.887398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:13.887398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:13.887398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:13.887398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:13.887398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 26/51] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:41:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:13.892727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69907ba9-a423-4463-9462-feb55104b6e6 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:13.989733 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69907ba9-a423-4463-9462-feb55104b6e6 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:13.998580 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:14.023088 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b18f5e-2525-4a91-856b-cf92b03b974e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:14.023693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b18f5e-2525-4a91-856b-cf92b03b974e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:14.023975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b18f5e-2525-4a91-856b-cf92b03b974e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:14.024278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b18f5e-2525-4a91-856b-cf92b03b974e None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:14.024936 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b18f5e-2525-4a91-856b-cf92b03b974e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:14.025239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b18f5e-2525-4a91-856b-cf92b03b974e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:14.026609 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b18f5e-2525-4a91-856b-cf92b03b974e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:14.038452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 26/52] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:14.044678 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966afc1-ebbd-4a3f-a16b-4ae1ea468be3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:14.078739 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:14.130680 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b18f5e-2525-4a91-856b-cf92b03b974e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:14.145783 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966afc1-ebbd-4a3f-a16b-4ae1ea468be3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:14.179359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20332237-dce4-47ba-98fa-13e44d947c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:14.179985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20332237-dce4-47ba-98fa-13e44d947c49 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:14.180291 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20332237-dce4-47ba-98fa-13e44d947c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:14.180567 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20332237-dce4-47ba-98fa-13e44d947c49 None admin] PATH_INFO: `/v3/users/984123d0eaef4929af01f39a46ebd51e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:14.181691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20332237-dce4-47ba-98fa-13e44d947c49 None admin] RBAC: Authorizing `identity:get_user(user_id=984123d0eaef4929af01f39a46ebd51e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:14.183150 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20332237-dce4-47ba-98fa-13e44d947c49 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:14.184977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 27/53] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:14 2022] GET /identity/v3/users/984123d0eaef4929af01f39a46ebd51e => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:14.190151 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20332237-dce4-47ba-98fa-13e44d947c49 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:14.231250 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b18f5e-2525-4a91-856b-cf92b03b974e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:14.264704 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09567fa-0b26-40b0-96fc-66d5dee9be5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:14.265335 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09567fa-0b26-40b0-96fc-66d5dee9be5c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:14.265646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09567fa-0b26-40b0-96fc-66d5dee9be5c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:14.265914 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09567fa-0b26-40b0-96fc-66d5dee9be5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:14.267027 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09567fa-0b26-40b0-96fc-66d5dee9be5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:14.269082 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09567fa-0b26-40b0-96fc-66d5dee9be5c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:14.270664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 27/54] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:14.275969 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09567fa-0b26-40b0-96fc-66d5dee9be5c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:14.288784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20332237-dce4-47ba-98fa-13e44d947c49 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:14.290310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:14.316691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:14.325119 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96082a1-add8-4fe8-8d88-6eb5c3b53a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:14.325749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96082a1-add8-4fe8-8d88-6eb5c3b53a86 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:14.326028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96082a1-add8-4fe8-8d88-6eb5c3b53a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:14.326302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96082a1-add8-4fe8-8d88-6eb5c3b53a86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:14.327418 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96082a1-add8-4fe8-8d88-6eb5c3b53a86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:14.328907 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96082a1-add8-4fe8-8d88-6eb5c3b53a86 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:14.330468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 28/55] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:14.336123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96082a1-add8-4fe8-8d88-6eb5c3b53a86 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:14.376149 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09567fa-0b26-40b0-96fc-66d5dee9be5c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:14.409692 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ce89e-861d-4c03-b167-2f060e4fcb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:14.410296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ce89e-861d-4c03-b167-2f060e4fcb31 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:14.410575 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ce89e-861d-4c03-b167-2f060e4fcb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:14.410848 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ce89e-861d-4c03-b167-2f060e4fcb31 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:14.411504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ce89e-861d-4c03-b167-2f060e4fcb31 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:14.414999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ce89e-861d-4c03-b167-2f060e4fcb31 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:14.439323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 28/56] 10.222.0.26 () {60 vars in 1140 bytes} [Wed Sep 7 07:41:14 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:41:14.439486 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96082a1-add8-4fe8-8d88-6eb5c3b53a86 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:14.476888 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f24b5-4d8b-46b7-bc2e-464b85eca33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:14.477545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f24b5-4d8b-46b7-bc2e-464b85eca33a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:14.477851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f24b5-4d8b-46b7-bc2e-464b85eca33a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:14.478131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f24b5-4d8b-46b7-bc2e-464b85eca33a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:14.478876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f24b5-4d8b-46b7-bc2e-464b85eca33a None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.domain.id=default, user.id=984123d0eaef4929af01f39a46ebd51e) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:14.479186 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f24b5-4d8b-46b7-bc2e-464b85eca33a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:14.480718 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f24b5-4d8b-46b7-bc2e-464b85eca33a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:14.488003 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 29/57] 10.222.0.26 () {56 vars in 1322 bytes} [Wed Sep 7 07:41:14 2022] GET /identity/v3/role_assignments?user.id=984123d0eaef4929af01f39a46ebd51e&scope.domain.id=default&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:16.610369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9b39e-fa7e-4314-9eab-35ea5979c893 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:16.610893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9b39e-fa7e-4314-9eab-35ea5979c893 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:16.610981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9b39e-fa7e-4314-9eab-35ea5979c893 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:16.612748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 29/58] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:16.617993 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3883e-2a38-4b0b-956f-02c3d7bd93ef None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:16.618404 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3883e-2a38-4b0b-956f-02c3d7bd93ef None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:16.618549 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3883e-2a38-4b0b-956f-02c3d7bd93ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:16.636850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a6c3883e-2a38-4b0b-956f-02c3d7bd93ef None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:16.638302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a6c3883e-2a38-4b0b-956f-02c3d7bd93ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:16.665287 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:16.704744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 30/59] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:16.981217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9b39e-fa7e-4314-9eab-35ea5979c893 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:17.070426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:17.081786 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9b39e-fa7e-4314-9eab-35ea5979c893 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:17.115087 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76980fb5-dbad-4c15-9961-75cee5f4790d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:17.115706 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76980fb5-dbad-4c15-9961-75cee5f4790d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:17.115986 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76980fb5-dbad-4c15-9961-75cee5f4790d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:17.116294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76980fb5-dbad-4c15-9961-75cee5f4790d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:17.117399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76980fb5-dbad-4c15-9961-75cee5f4790d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:17.118833 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76980fb5-dbad-4c15-9961-75cee5f4790d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:17.120402 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 30/60] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:17.125779 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3883e-2a38-4b0b-956f-02c3d7bd93ef None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:17.231480 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3883e-2a38-4b0b-956f-02c3d7bd93ef None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:17.266430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f1518-38a8-4d4c-9d0e-0b2eb951c70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:17.267030 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f1518-38a8-4d4c-9d0e-0b2eb951c70a None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:17.267309 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f1518-38a8-4d4c-9d0e-0b2eb951c70a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:17.267579 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f1518-38a8-4d4c-9d0e-0b2eb951c70a None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:17.268417 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f1518-38a8-4d4c-9d0e-0b2eb951c70a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:17.269867 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f1518-38a8-4d4c-9d0e-0b2eb951c70a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:17.295768 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 31/61] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:41:17 2022] POST /identity/v3/users => generated 275 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:41:20.663857 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadae89d-fe1e-49d0-af28-bf248bde7746 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:20.664352 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadae89d-fe1e-49d0-af28-bf248bde7746 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:20.664352 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadae89d-fe1e-49d0-af28-bf248bde7746 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:20.665695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 31/62] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:20.667910 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea36535-77aa-42fd-a086-2e46f3606bb0 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:20.668698 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea36535-77aa-42fd-a086-2e46f3606bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:20.669166 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea36535-77aa-42fd-a086-2e46f3606bb0 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:20.671021 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 32/63] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:20.671409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d70225-b2d6-4af7-865a-e73882d55c4a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:20.671700 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d70225-b2d6-4af7-865a-e73882d55c4a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:20.671984 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d70225-b2d6-4af7-865a-e73882d55c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:20.677057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60ded1-b782-4bd5-ad67-37aa851903ad None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:20.677594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60ded1-b782-4bd5-ad67-37aa851903ad None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:20.678105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60ded1-b782-4bd5-ad67-37aa851903ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:20.697692 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-45d70225-b2d6-4af7-865a-e73882d55c4a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:20.700358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-45d70225-b2d6-4af7-865a-e73882d55c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:20.706904 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3d60ded1-b782-4bd5-ad67-37aa851903ad None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:20.709548 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3d60ded1-b782-4bd5-ad67-37aa851903ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:20.736507 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:20.811798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 33/64] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:20.822451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 32/65] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:21.077076 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ecd7e-9032-4d46-b65c-bceb589769fe None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:21.077425 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ecd7e-9032-4d46-b65c-bceb589769fe None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:21.077597 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ecd7e-9032-4d46-b65c-bceb589769fe None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:21.078910 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 34/66] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:21.086014 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02704de3-81f0-4618-9066-355c2af17b68 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:21.086306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02704de3-81f0-4618-9066-355c2af17b68 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:21.086595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02704de3-81f0-4618-9066-355c2af17b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:21.088979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ca09e-b54f-45f6-bd6b-c4d7431a2321 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:21.089266 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ca09e-b54f-45f6-bd6b-c4d7431a2321 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:21.089676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ca09e-b54f-45f6-bd6b-c4d7431a2321 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:21.091409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 35/67] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:21.096456 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96dbd0-80f8-436b-9cf0-5efbcb98bc91 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:21.096750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96dbd0-80f8-436b-9cf0-5efbcb98bc91 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:21.097026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96dbd0-80f8-436b-9cf0-5efbcb98bc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:21.115903 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-4b96dbd0-80f8-436b-9cf0-5efbcb98bc91 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:21.117170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4b96dbd0-80f8-436b-9cf0-5efbcb98bc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:21.117278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-02704de3-81f0-4618-9066-355c2af17b68 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:21.118369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-02704de3-81f0-4618-9066-355c2af17b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:21.191896 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 36/68] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:21.220039 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 33/69] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:21.240244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96dbd0-80f8-436b-9cf0-5efbcb98bc91 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:21.248956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b8d5e-73b6-47f9-a282-54933fea6bb5 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:21.249231 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b8d5e-73b6-47f9-a282-54933fea6bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:21.249508 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b8d5e-73b6-47f9-a282-54933fea6bb5 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:21.250841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 34/70] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:41:21.257300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f32886-e7e8-4fe8-afa4-d372f508ca64 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:21.257582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f32886-e7e8-4fe8-afa4-d372f508ca64 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:21.257859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f32886-e7e8-4fe8-afa4-d372f508ca64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:21.276183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-93f32886-e7e8-4fe8-afa4-d372f508ca64 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:21.277465 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-93f32886-e7e8-4fe8-afa4-d372f508ca64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:21.292930 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:21.344574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:21.350746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 35/71] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:21.356298 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f32886-e7e8-4fe8-afa4-d372f508ca64 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:21.367231 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96dbd0-80f8-436b-9cf0-5efbcb98bc91 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:21.405657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fc281-c895-46da-9de2-dbc15d4338a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:21.406294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fc281-c895-46da-9de2-dbc15d4338a7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:21.406739 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fc281-c895-46da-9de2-dbc15d4338a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:21.407607 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fc281-c895-46da-9de2-dbc15d4338a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:21.419553 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4fc281-c895-46da-9de2-dbc15d4338a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:21.425741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-9b4fc281-c895-46da-9de2-dbc15d4338a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:21.425741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:21.425741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:21.425741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:21.425741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:21.425741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:21.425741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:21.425741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:21.425741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:21.425741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:21.425741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:21.425741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:21.425741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:21.425741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:21.425741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:21.425741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:21.426773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:21.426773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:21.426773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:21.426773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:21.426773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:21.426773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:21.426773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:21.426773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:21.426773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:21.426773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:21.426773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:21.426773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:21.426773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:21.426773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:21.426773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:21.426773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:21.427683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:21.427683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:21.427683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:21.427683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:21.427683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:21.427683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:21.427683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:21.427683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:21.427683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:21.427683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:21.427683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:21.427683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:21.429654 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 37/72] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:21.435159 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fc281-c895-46da-9de2-dbc15d4338a7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:21.552803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f32886-e7e8-4fe8-afa4-d372f508ca64 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:21.554363 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fc281-c895-46da-9de2-dbc15d4338a7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:21.588210 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fcae0-e9db-4d81-a264-02d13510528a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:21.588898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fcae0-e9db-4d81-a264-02d13510528a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:21.589177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fcae0-e9db-4d81-a264-02d13510528a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:21.589449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fcae0-e9db-4d81-a264-02d13510528a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:21.592634 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec9c3b-cfed-46a8-bec0-30379e1761c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:21.593240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9c3b-cfed-46a8-bec0-30379e1761c5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:21.593522 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9c3b-cfed-46a8-bec0-30379e1761c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:21.593798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9c3b-cfed-46a8-bec0-30379e1761c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:21.594457 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec9c3b-cfed-46a8-bec0-30379e1761c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:21.594818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec9c3b-cfed-46a8-bec0-30379e1761c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:21.596779 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec9c3b-cfed-46a8-bec0-30379e1761c5 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:21.600845 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1fcae0-e9db-4d81-a264-02d13510528a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:21.609935 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 38/73] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:21.610079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ec1fcae0-e9db-4d81-a264-02d13510528a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:21.610079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:21.610079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:21.610079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:21.610079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:21.610079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:21.610079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:21.610079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:21.610079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:21.610079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:21.610079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:21.610079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:21.610079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:21.610079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:21.610079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:21.610079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:21.610875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:21.610875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:21.610875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:21.610875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:21.610875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:21.610875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:21.610875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:21.610875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:21.610875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:21.610875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:21.610875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:21.610875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:21.610875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:21.610875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:21.610875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:21.610875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:21.612162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:21.612162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:21.612162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:21.612162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:21.612162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:21.612162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:21.612162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:21.612162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:21.612162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:21.612162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:21.612162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:21.612162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:21.612889 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec9c3b-cfed-46a8-bec0-30379e1761c5 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:21.612957 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 36/74] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:41:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:21.612957 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fcae0-e9db-4d81-a264-02d13510528a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:21.616242 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:21.623046 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:21.720992 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec9c3b-cfed-46a8-bec0-30379e1761c5 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:21.723487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fcae0-e9db-4d81-a264-02d13510528a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:21.756487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c64b4-400b-42c4-88ce-ab2519d038f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:21.756852 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42846bb5-4a0b-4491-b778-0a71d59dbb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:21.757114 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c64b4-400b-42c4-88ce-ab2519d038f9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:21.757398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c64b4-400b-42c4-88ce-ab2519d038f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:21.757485 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42846bb5-4a0b-4491-b778-0a71d59dbb71 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:21.757674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c64b4-400b-42c4-88ce-ab2519d038f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:21.757757 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42846bb5-4a0b-4491-b778-0a71d59dbb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:21.758001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42846bb5-4a0b-4491-b778-0a71d59dbb71 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:21.768330 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c64b4-400b-42c4-88ce-ab2519d038f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:21.768935 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42846bb5-4a0b-4491-b778-0a71d59dbb71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:21.773782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-42846bb5-4a0b-4491-b778-0a71d59dbb71 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:41:21.773782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:21.773782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:21.773782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:21.773782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:21.773782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:21.773782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:21.773782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:21.773782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:21.773782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:21.773782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:21.773782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:21.773782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:21.773782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:21.773782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:21.773782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:21.774601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:21.774601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:21.774601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:21.774601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:21.774601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:21.774601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:21.774601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:21.774601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:21.774601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:21.774601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:21.774601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:21.774601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:21.774601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:21.774601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:21.774601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:21.774601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:21.775328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:21.775328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:21.775328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:21.775328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:21.775328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:21.775328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:21.775328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:21.775328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:21.775328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:21.775328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:21.775328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:41:21.775328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:21.775895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-7f4c64b4-400b-42c4-88ce-ab2519d038f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:21.775895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:21.775895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:21.775895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:21.775895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:21.775895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:21.775895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:21.775895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:21.775895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:21.775895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:21.775895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:21.775895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:21.775895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:21.775895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:21.775895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:21.775895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:21.776819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:21.776819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:21.776819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:21.776819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:21.776819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:21.776819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:21.776819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:21.776819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:21.776819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:21.776819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:21.776819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:21.776819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:21.776819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:21.776819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:21.776819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:21.776819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:21.777583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:21.777583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:21.777583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:21.777583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:21.777583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:21.777583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:21.777583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:21.777583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:21.777583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:21.777583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:21.777583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:21.777583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:21.781951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 37/75] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:41:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:21.781951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42846bb5-4a0b-4491-b778-0a71d59dbb71 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:21.782077 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 39/76] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:21.782077 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c64b4-400b-42c4-88ce-ab2519d038f9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:21.840184 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:21.841795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:21.876918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42846bb5-4a0b-4491-b778-0a71d59dbb71 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:21.888638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c64b4-400b-42c4-88ce-ab2519d038f9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:21.911907 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e1a4f-2dcb-4bf5-8272-a1c0dfc91b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:21.912540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e1a4f-2dcb-4bf5-8272-a1c0dfc91b8c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:21.912822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e1a4f-2dcb-4bf5-8272-a1c0dfc91b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:21.913094 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e1a4f-2dcb-4bf5-8272-a1c0dfc91b8c None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:21.913752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e1a4f-2dcb-4bf5-8272-a1c0dfc91b8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:21.914059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e1a4f-2dcb-4bf5-8272-a1c0dfc91b8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:21.915468 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e1a4f-2dcb-4bf5-8272-a1c0dfc91b8c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:21.922163 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3fa8f4-6b87-4978-a1b7-f91227788f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:21.922813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fa8f4-6b87-4978-a1b7-f91227788f7a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:21.923092 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fa8f4-6b87-4978-a1b7-f91227788f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:21.923300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 38/77] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:21.923420 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fa8f4-6b87-4978-a1b7-f91227788f7a None admin] PATH_INFO: `/v3/users/3d9981f1b2c1421b8d5a89485cc87def` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:21.924633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e1a4f-2dcb-4bf5-8272-a1c0dfc91b8c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:21.937530 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3fa8f4-6b87-4978-a1b7-f91227788f7a None admin] RBAC: Authorizing `identity:get_user(user_id=3d9981f1b2c1421b8d5a89485cc87def)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:21.938986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3fa8f4-6b87-4978-a1b7-f91227788f7a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:21.941382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 40/78] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:21 2022] GET /identity/v3/users/3d9981f1b2c1421b8d5a89485cc87def => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:21.942143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3fa8f4-6b87-4978-a1b7-f91227788f7a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:22.024657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e1a4f-2dcb-4bf5-8272-a1c0dfc91b8c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:22.040500 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3fa8f4-6b87-4978-a1b7-f91227788f7a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:22.052799 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:22.059016 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f780c-7f92-4e31-8105-d2184873ad19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:22.059636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f780c-7f92-4e31-8105-d2184873ad19 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:22.059914 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f780c-7f92-4e31-8105-d2184873ad19 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:22.060219 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f780c-7f92-4e31-8105-d2184873ad19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:22.061137 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:22.061331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f780c-7f92-4e31-8105-d2184873ad19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:22.062880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f780c-7f92-4e31-8105-d2184873ad19 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:22.064562 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 39/79] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:22.068830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f780c-7f92-4e31-8105-d2184873ad19 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:22.079237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386409c-3e83-4b65-a8f8-492ffe0fcc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:22.079847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386409c-3e83-4b65-a8f8-492ffe0fcc60 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:22.080167 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386409c-3e83-4b65-a8f8-492ffe0fcc60 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:22.080491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386409c-3e83-4b65-a8f8-492ffe0fcc60 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:22.081340 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386409c-3e83-4b65-a8f8-492ffe0fcc60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:22.081649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386409c-3e83-4b65-a8f8-492ffe0fcc60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:22.083046 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386409c-3e83-4b65-a8f8-492ffe0fcc60 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:22.095346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 41/80] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:41:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:22.095346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386409c-3e83-4b65-a8f8-492ffe0fcc60 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:22.165320 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f780c-7f92-4e31-8105-d2184873ad19 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:22.191947 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386409c-3e83-4b65-a8f8-492ffe0fcc60 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:22.198148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51797294-31eb-4bb1-b385-1732647581f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:22.198750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51797294-31eb-4bb1-b385-1732647581f0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:22.199029 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51797294-31eb-4bb1-b385-1732647581f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:22.199301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51797294-31eb-4bb1-b385-1732647581f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:22.199950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51797294-31eb-4bb1-b385-1732647581f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:22.200288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51797294-31eb-4bb1-b385-1732647581f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:22.201642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51797294-31eb-4bb1-b385-1732647581f0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:22.209904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 40/81] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:22.210504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51797294-31eb-4bb1-b385-1732647581f0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:22.228543 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56aa3fa-5aa0-4a42-bc20-dcbae2b74d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:22.229164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56aa3fa-5aa0-4a42-bc20-dcbae2b74d1b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:22.229444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56aa3fa-5aa0-4a42-bc20-dcbae2b74d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:22.229716 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56aa3fa-5aa0-4a42-bc20-dcbae2b74d1b None admin] PATH_INFO: `/v3/users/984123d0eaef4929af01f39a46ebd51e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:22.230834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56aa3fa-5aa0-4a42-bc20-dcbae2b74d1b None admin] RBAC: Authorizing `identity:get_user(user_id=984123d0eaef4929af01f39a46ebd51e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:22.232308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56aa3fa-5aa0-4a42-bc20-dcbae2b74d1b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:22.233874 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 42/82] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:22 2022] GET /identity/v3/users/984123d0eaef4929af01f39a46ebd51e => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:22.234594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56aa3fa-5aa0-4a42-bc20-dcbae2b74d1b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:22.269271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:22.285095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:22.312336 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51797294-31eb-4bb1-b385-1732647581f0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:22.335440 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56aa3fa-5aa0-4a42-bc20-dcbae2b74d1b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:22.346221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3d3bf-bc6e-4c34-856d-a8a548c9f1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:22.346902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3d3bf-bc6e-4c34-856d-a8a548c9f1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:22.347183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3d3bf-bc6e-4c34-856d-a8a548c9f1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:22.347457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3d3bf-bc6e-4c34-856d-a8a548c9f1b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:22.367703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac04bfa-70e0-49ac-9686-38d5f2095351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:22.368329 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac04bfa-70e0-49ac-9686-38d5f2095351 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:22.368608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac04bfa-70e0-49ac-9686-38d5f2095351 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:22.368883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac04bfa-70e0-49ac-9686-38d5f2095351 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:22.369521 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac04bfa-70e0-49ac-9686-38d5f2095351 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:22.370965 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac04bfa-70e0-49ac-9686-38d5f2095351 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:22.392783 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 43/83] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:41:22 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:41:22.394174 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac04bfa-70e0-49ac-9686-38d5f2095351 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:22.436981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3d3bf-bc6e-4c34-856d-a8a548c9f1b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:22.438700 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3d3bf-bc6e-4c34-856d-a8a548c9f1b8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:22.510632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:22.512316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac04bfa-70e0-49ac-9686-38d5f2095351 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:22.521068 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:22.559397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2391be-0cf5-47f9-9f9c-2c99b9378d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:22.560045 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2391be-0cf5-47f9-9f9c-2c99b9378d27 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:22.560383 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2391be-0cf5-47f9-9f9c-2c99b9378d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:22.560665 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2391be-0cf5-47f9-9f9c-2c99b9378d27 None admin] PATH_INFO: `/v3/users/3d9981f1b2c1421b8d5a89485cc87def` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:22.561834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2391be-0cf5-47f9-9f9c-2c99b9378d27 None admin] RBAC: Authorizing `identity:get_user(user_id=3d9981f1b2c1421b8d5a89485cc87def)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:22.563329 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2391be-0cf5-47f9-9f9c-2c99b9378d27 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:22.565678 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 44/84] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:22 2022] GET /identity/v3/users/3d9981f1b2c1421b8d5a89485cc87def => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:22.566753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2391be-0cf5-47f9-9f9c-2c99b9378d27 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:22.570724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 41/85] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:41:22.571736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3d3bf-bc6e-4c34-856d-a8a548c9f1b8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:22.689237 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3d3bf-bc6e-4c34-856d-a8a548c9f1b8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:22.695766 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2391be-0cf5-47f9-9f9c-2c99b9378d27 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:22.731026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da2e3b-3fe6-4606-b918-11d678fbc0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:22.731699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da2e3b-3fe6-4606-b918-11d678fbc0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:22.731985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da2e3b-3fe6-4606-b918-11d678fbc0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:22.732358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da2e3b-3fe6-4606-b918-11d678fbc0e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:22.745640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d94ad-69d6-4e83-804d-1e76c22b930a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:22.746386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d94ad-69d6-4e83-804d-1e76c22b930a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:22.746672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d94ad-69d6-4e83-804d-1e76c22b930a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:22.746950 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d94ad-69d6-4e83-804d-1e76c22b930a None admin] PATH_INFO: `/v3/users/3d9981f1b2c1421b8d5a89485cc87def` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:22.748129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d94ad-69d6-4e83-804d-1e76c22b930a None admin] RBAC: Authorizing `identity:get_user(user_id=3d9981f1b2c1421b8d5a89485cc87def)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:22.749594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d94ad-69d6-4e83-804d-1e76c22b930a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:22.751198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 45/86] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:22 2022] GET /identity/v3/users/3d9981f1b2c1421b8d5a89485cc87def => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:22.752000 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d94ad-69d6-4e83-804d-1e76c22b930a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:22.766315 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:22.809490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da2e3b-3fe6-4606-b918-11d678fbc0e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:22.811199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da2e3b-3fe6-4606-b918-11d678fbc0e1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:22.811674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:22.868281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d94ad-69d6-4e83-804d-1e76c22b930a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:22.905691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b200e87-3671-45a8-b940-8542d707749d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:22.906335 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b200e87-3671-45a8-b940-8542d707749d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:22.906675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b200e87-3671-45a8-b940-8542d707749d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:22.906885 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b200e87-3671-45a8-b940-8542d707749d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:22.915859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 42/87] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:41:22.916931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da2e3b-3fe6-4606-b918-11d678fbc0e1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:22.981255 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:22.992611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b200e87-3671-45a8-b940-8542d707749d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:22.994565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b200e87-3671-45a8-b940-8542d707749d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:23.027288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da2e3b-3fe6-4606-b918-11d678fbc0e1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:23.041865 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:23.068732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aec3f2-6b65-4a80-8d6c-1341e6326e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:23.069188 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aec3f2-6b65-4a80-8d6c-1341e6326e0d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:23.069473 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aec3f2-6b65-4a80-8d6c-1341e6326e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:23.069755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aec3f2-6b65-4a80-8d6c-1341e6326e0d None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:23.070906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aec3f2-6b65-4a80-8d6c-1341e6326e0d None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:23.072504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aec3f2-6b65-4a80-8d6c-1341e6326e0d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:23.074268 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 43/88] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:22 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:23.075033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aec3f2-6b65-4a80-8d6c-1341e6326e0d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:23.107091 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 46/89] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:41:23.108328 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b200e87-3671-45a8-b940-8542d707749d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:23.191541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aec3f2-6b65-4a80-8d6c-1341e6326e0d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:23.218953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b200e87-3671-45a8-b940-8542d707749d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:23.221815 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:23.230205 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd6a58-7373-4a38-87d3-ee589c360c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:23.231203 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd6a58-7373-4a38-87d3-ee589c360c25 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:23.232155 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd6a58-7373-4a38-87d3-ee589c360c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:23.232267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd6a58-7373-4a38-87d3-ee589c360c25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:23.259513 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cd85f-1f1e-4014-832d-babe8b8b93a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:23.260537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cd85f-1f1e-4014-832d-babe8b8b93a7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:23.260964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cd85f-1f1e-4014-832d-babe8b8b93a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:23.261386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cd85f-1f1e-4014-832d-babe8b8b93a7 None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:23.262701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2cd85f-1f1e-4014-832d-babe8b8b93a7 None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:23.264396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2cd85f-1f1e-4014-832d-babe8b8b93a7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:23.266433 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 47/90] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:23 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:23.267363 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cd85f-1f1e-4014-832d-babe8b8b93a7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:23.287489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:23.317907 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd6a58-7373-4a38-87d3-ee589c360c25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:23.320703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd6a58-7373-4a38-87d3-ee589c360c25 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:23.414030 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cd85f-1f1e-4014-832d-babe8b8b93a7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:23.454941 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2f7cf-3dc9-407e-82a1-48564d8ef2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:23.456104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2f7cf-3dc9-407e-82a1-48564d8ef2ae None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:23.456544 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2f7cf-3dc9-407e-82a1-48564d8ef2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:23.456944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2f7cf-3dc9-407e-82a1-48564d8ef2ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:23.457370 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 44/91] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:41:23.457553 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2f7cf-3dc9-407e-82a1-48564d8ef2ae None admin] RBAC: Adding query filter params (role.id=6ebbed85905048a198e4261a3510dafc, scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5, user.id=3d9981f1b2c1421b8d5a89485cc87def) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:23.457853 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2f7cf-3dc9-407e-82a1-48564d8ef2ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:23.458837 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd6a58-7373-4a38-87d3-ee589c360c25 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:23.459322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2f7cf-3dc9-407e-82a1-48564d8ef2ae None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:23.465364 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 48/92] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:23 2022] GET /identity/v3/role_assignments?user.id=3d9981f1b2c1421b8d5a89485cc87def&scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5&role.id=6ebbed85905048a198e4261a3510dafc => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:23.466167 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2f7cf-3dc9-407e-82a1-48564d8ef2ae None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:23.492295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:23.550682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:23.575396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd6a58-7373-4a38-87d3-ee589c360c25 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:23.582246 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2f7cf-3dc9-407e-82a1-48564d8ef2ae None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:23.617407 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ac898-05b9-40ab-939c-4e45014501ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:23.618554 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ac898-05b9-40ab-939c-4e45014501ca None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:23.619086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ac898-05b9-40ab-939c-4e45014501ca None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:23.619557 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ac898-05b9-40ab-939c-4e45014501ca None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:23.621141 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ac898-05b9-40ab-939c-4e45014501ca None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:23.621939 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2da25-1308-4791-a9cf-5271dbf6d19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:23.622555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2da25-1308-4791-a9cf-5271dbf6d19c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:23.622841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2da25-1308-4791-a9cf-5271dbf6d19c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:23.623116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2da25-1308-4791-a9cf-5271dbf6d19c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:23.623778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ac898-05b9-40ab-939c-4e45014501ca None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:23.623880 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2da25-1308-4791-a9cf-5271dbf6d19c None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5, user.id=984123d0eaef4929af01f39a46ebd51e) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:23.624172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2da25-1308-4791-a9cf-5271dbf6d19c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:23.625621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2da25-1308-4791-a9cf-5271dbf6d19c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:23.626209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 45/93] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:23 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:23.627455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ac898-05b9-40ab-939c-4e45014501ca None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:23.632649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 49/94] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:23 2022] GET /identity/v3/role_assignments?user.id=984123d0eaef4929af01f39a46ebd51e&scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:23.633071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2da25-1308-4791-a9cf-5271dbf6d19c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:23.743628 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ac898-05b9-40ab-939c-4e45014501ca None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:23.780343 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:23.785038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35721b1c-5de4-4d6b-a8b2-b5ef9d4dd269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:23.786425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35721b1c-5de4-4d6b-a8b2-b5ef9d4dd269 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:23.786915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35721b1c-5de4-4d6b-a8b2-b5ef9d4dd269 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:23.787391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35721b1c-5de4-4d6b-a8b2-b5ef9d4dd269 None admin] PATH_INFO: `/v3/projects/bfab41741dcd4147ae89996789a3dbaa` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:23.788967 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2da25-1308-4791-a9cf-5271dbf6d19c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:23.789643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35721b1c-5de4-4d6b-a8b2-b5ef9d4dd269 None admin] RBAC: Authorizing `identity:get_project(project_id=bfab41741dcd4147ae89996789a3dbaa)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:23.791678 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35721b1c-5de4-4d6b-a8b2-b5ef9d4dd269 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:23.793726 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 46/95] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:23 2022] GET /identity/v3/projects/bfab41741dcd4147ae89996789a3dbaa => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:23.800960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35721b1c-5de4-4d6b-a8b2-b5ef9d4dd269 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:23.884931 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7cceb4-d87d-462c-83a8-65c2a650623c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:23.885680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7cceb4-d87d-462c-83a8-65c2a650623c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:23.885895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7cceb4-d87d-462c-83a8-65c2a650623c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:23.886171 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7cceb4-d87d-462c-83a8-65c2a650623c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:23.886909 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7cceb4-d87d-462c-83a8-65c2a650623c None admin] RBAC: Adding query filter params (role.id=84a30faea3e94eb48b03ec7ceb7c1392, scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5, user.id=3d9981f1b2c1421b8d5a89485cc87def) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:23.887225 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7cceb4-d87d-462c-83a8-65c2a650623c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:23.888726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7cceb4-d87d-462c-83a8-65c2a650623c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:23.895845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 50/96] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:23 2022] GET /identity/v3/role_assignments?user.id=3d9981f1b2c1421b8d5a89485cc87def&scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5&role.id=84a30faea3e94eb48b03ec7ceb7c1392 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:23.948453 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35721b1c-5de4-4d6b-a8b2-b5ef9d4dd269 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:23.993171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ac7a1-53a2-4970-9c4c-51ffa32e14f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:23.994260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ac7a1-53a2-4970-9c4c-51ffa32e14f4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:23.994890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ac7a1-53a2-4970-9c4c-51ffa32e14f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:23.995369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ac7a1-53a2-4970-9c4c-51ffa32e14f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:23.996354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ac7a1-53a2-4970-9c4c-51ffa32e14f4 None admin] RBAC: Adding query filter params (role.id=6ebbed85905048a198e4261a3510dafc, scope.project.id=bfab41741dcd4147ae89996789a3dbaa, user.id=3d9981f1b2c1421b8d5a89485cc87def) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:23.996903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ac7a1-53a2-4970-9c4c-51ffa32e14f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:23.998574 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ac7a1-53a2-4970-9c4c-51ffa32e14f4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:24.013345 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 47/97] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:23 2022] GET /identity/v3/role_assignments?user.id=3d9981f1b2c1421b8d5a89485cc87def&scope.project.id=bfab41741dcd4147ae89996789a3dbaa&role.id=6ebbed85905048a198e4261a3510dafc => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:24.917406 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bf0df-59e7-40ba-938b-5581a0e4c52b None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:24.917910 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bf0df-59e7-40ba-938b-5581a0e4c52b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:24.918013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bf0df-59e7-40ba-938b-5581a0e4c52b None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:24.919494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 51/98] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:24.933775 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f31966-405e-40c3-8ff7-3e6c731039fe None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:24.934685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f31966-405e-40c3-8ff7-3e6c731039fe None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:24.935186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f31966-405e-40c3-8ff7-3e6c731039fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:24.965116 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c8f31966-405e-40c3-8ff7-3e6c731039fe None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:24.966334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c8f31966-405e-40c3-8ff7-3e6c731039fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:24.991868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:25.046943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 48/99] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:25.416261 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bf0df-59e7-40ba-938b-5581a0e4c52b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:25.419187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126af9c9-2421-49cb-879a-eeed4602877f None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:25.419475 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126af9c9-2421-49cb-879a-eeed4602877f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:25.419758 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126af9c9-2421-49cb-879a-eeed4602877f None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:25.419880 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:25.421509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 49/100] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:25.432962 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a090bf-e3da-43bf-9364-cf58803758a2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:25.433245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a090bf-e3da-43bf-9364-cf58803758a2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:25.433801 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a090bf-e3da-43bf-9364-cf58803758a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:25.464884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-18a090bf-e3da-43bf-9364-cf58803758a2 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:25.466062 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-18a090bf-e3da-43bf-9364-cf58803758a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:25.570068 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bf0df-59e7-40ba-938b-5581a0e4c52b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:25.626314 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8e9cc-50d3-4ac4-8e48-cd5a91373b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:25.627465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8e9cc-50d3-4ac4-8e48-cd5a91373b1e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:25.627972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8e9cc-50d3-4ac4-8e48-cd5a91373b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:25.628474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8e9cc-50d3-4ac4-8e48-cd5a91373b1e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:25.642030 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 50/101] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:25.652684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8e9cc-50d3-4ac4-8e48-cd5a91373b1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:25.658462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-40c8e9cc-50d3-4ac4-8e48-cd5a91373b1e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:41:25.658462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:25.658462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:25.658462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:25.658462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:25.658462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:25.658462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:25.658462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:25.658462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:25.658462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:25.658462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:25.658462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:25.658462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:25.658462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:25.658462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:25.658462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:25.659501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:25.659501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:25.659501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:25.659501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:25.659501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:25.659501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:25.659501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:25.659501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:25.659501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:25.659501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:25.659501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:25.659501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:25.659501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:25.659501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:25.659501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:25.659501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:25.660260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:25.660260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:25.660260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:25.660260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:25.660260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:25.660260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:25.660260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:25.660260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:25.660260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:25.660260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:25.660260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:41:25.660260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:25.662740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 52/102] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:25.668788 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a090bf-e3da-43bf-9364-cf58803758a2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:25.761848 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:25.778454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a090bf-e3da-43bf-9364-cf58803758a2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:25.819707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0671cb-3fe2-4780-b657-5824bdf17173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:25.820710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0671cb-3fe2-4780-b657-5824bdf17173 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:25.821190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0671cb-3fe2-4780-b657-5824bdf17173 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:25.821681 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0671cb-3fe2-4780-b657-5824bdf17173 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:25.822607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0671cb-3fe2-4780-b657-5824bdf17173 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:25.823109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0671cb-3fe2-4780-b657-5824bdf17173 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:25.824787 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0671cb-3fe2-4780-b657-5824bdf17173 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:25.838233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 51/103] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:25.844262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8e9cc-50d3-4ac4-8e48-cd5a91373b1e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:25.917965 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:25.967459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8e9cc-50d3-4ac4-8e48-cd5a91373b1e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:26.002365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686038c-3368-40a3-8f30-336ea0b9474e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:26.002991 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686038c-3368-40a3-8f30-336ea0b9474e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:26.003272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686038c-3368-40a3-8f30-336ea0b9474e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:26.003550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686038c-3368-40a3-8f30-336ea0b9474e None admin] PATH_INFO: `/v3/users/a9894cf5ac414bcfa637feb6d3fd1943` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:26.016907 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686038c-3368-40a3-8f30-336ea0b9474e None admin] RBAC: Authorizing `identity:get_user(user_id=a9894cf5ac414bcfa637feb6d3fd1943)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:26.018430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686038c-3368-40a3-8f30-336ea0b9474e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:26.020281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 53/104] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:25 2022] GET /identity/v3/users/a9894cf5ac414bcfa637feb6d3fd1943 => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:26.029153 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0671cb-3fe2-4780-b657-5824bdf17173 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:26.156510 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0671cb-3fe2-4780-b657-5824bdf17173 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:26.201005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d3a5b-06bb-4be7-9d4d-17392d7d4ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:26.201946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d3a5b-06bb-4be7-9d4d-17392d7d4ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:26.202433 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d3a5b-06bb-4be7-9d4d-17392d7d4ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:26.202927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d3a5b-06bb-4be7-9d4d-17392d7d4ec1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:26.226629 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:26.291896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d3a5b-06bb-4be7-9d4d-17392d7d4ec1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:26.293796 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d3a5b-06bb-4be7-9d4d-17392d7d4ec1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:26.326889 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686038c-3368-40a3-8f30-336ea0b9474e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:26.527401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 52/105] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:41:26.534987 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d3a5b-06bb-4be7-9d4d-17392d7d4ec1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:26.570555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686038c-3368-40a3-8f30-336ea0b9474e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:26.600646 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:26.633718 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea0b88-c2f7-49d8-beb4-34c3ed475e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:26.634371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0b88-c2f7-49d8-beb4-34c3ed475e60 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:26.634655 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0b88-c2f7-49d8-beb4-34c3ed475e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:26.634932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0b88-c2f7-49d8-beb4-34c3ed475e60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:26.636587 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:26.636775 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea0b88-c2f7-49d8-beb4-34c3ed475e60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:26.638223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea0b88-c2f7-49d8-beb4-34c3ed475e60 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:26.639855 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 54/106] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:26.646517 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea0b88-c2f7-49d8-beb4-34c3ed475e60 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:26.766385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d3a5b-06bb-4be7-9d4d-17392d7d4ec1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:26.829010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea0b88-c2f7-49d8-beb4-34c3ed475e60 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:26.856333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b31964-63d4-4f70-b6ec-99694fa930c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:26.856977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b31964-63d4-4f70-b6ec-99694fa930c1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:26.857258 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b31964-63d4-4f70-b6ec-99694fa930c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:26.857533 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b31964-63d4-4f70-b6ec-99694fa930c1 None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:26.858687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b31964-63d4-4f70-b6ec-99694fa930c1 None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:26.865729 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b31964-63d4-4f70-b6ec-99694fa930c1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:26.867536 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 53/107] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:26 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:26.876525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b31964-63d4-4f70-b6ec-99694fa930c1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:26.893176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78298096-2c4c-4758-b3a4-b3ca20e5ab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:26.893737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78298096-2c4c-4758-b3a4-b3ca20e5ab92 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:26.894016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78298096-2c4c-4758-b3a4-b3ca20e5ab92 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:26.894297 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78298096-2c4c-4758-b3a4-b3ca20e5ab92 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:26.894973 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78298096-2c4c-4758-b3a4-b3ca20e5ab92 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:26.896471 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78298096-2c4c-4758-b3a4-b3ca20e5ab92 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:26.944829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 55/108] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 7 07:41:26 2022] POST /identity/v3/projects => generated 302 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:41:27.069394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea4812-ffd0-40ca-aedb-fe264dfef88c None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:27.069808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea4812-ffd0-40ca-aedb-fe264dfef88c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:27.070076 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea4812-ffd0-40ca-aedb-fe264dfef88c None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:27.071561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 56/109] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:41:27.073295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b31964-63d4-4f70-b6ec-99694fa930c1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:27.077079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798aaf85-2386-4de7-8f2e-0e3d57afe118 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:27.077360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798aaf85-2386-4de7-8f2e-0e3d57afe118 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:27.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798aaf85-2386-4de7-8f2e-0e3d57afe118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:27.101216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-798aaf85-2386-4de7-8f2e-0e3d57afe118 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:27.102408 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-798aaf85-2386-4de7-8f2e-0e3d57afe118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:27.107957 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:27.141733 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342f330-b4f4-4104-a5a8-778a96e04142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:27.142647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342f330-b4f4-4104-a5a8-778a96e04142 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:27.142932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342f330-b4f4-4104-a5a8-778a96e04142 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:27.143203 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342f330-b4f4-4104-a5a8-778a96e04142 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:27.143951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342f330-b4f4-4104-a5a8-778a96e04142 None admin] RBAC: Adding query filter params (role.id=a5350d487c374ea680a13ac3044168fc, scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5, user.id=a9894cf5ac414bcfa637feb6d3fd1943) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:27.144279 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342f330-b4f4-4104-a5a8-778a96e04142 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:27.145866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342f330-b4f4-4104-a5a8-778a96e04142 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:27.152672 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 54/110] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:26 2022] GET /identity/v3/role_assignments?user.id=a9894cf5ac414bcfa637feb6d3fd1943&scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5&role.id=a5350d487c374ea680a13ac3044168fc => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:27.154433 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44232b10-c753-4255-ab60-35acd24b3d92 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:27.154541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44232b10-c753-4255-ab60-35acd24b3d92 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:27.154821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44232b10-c753-4255-ab60-35acd24b3d92 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:27.156301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 55/111] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:27.164440 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78a931-b1da-4bc1-a2a5-eb0cc091ce70 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:27.164895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78a931-b1da-4bc1-a2a5-eb0cc091ce70 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:27.165284 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78a931-b1da-4bc1-a2a5-eb0cc091ce70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:27.180034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:27.189777 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0d78a931-b1da-4bc1-a2a5-eb0cc091ce70 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:27.192502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0d78a931-b1da-4bc1-a2a5-eb0cc091ce70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:27.257410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 57/112] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:27.296379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 56/113] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:27.315159 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b4024-679e-4484-afa0-3e572a46a04a None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:27.315907 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b4024-679e-4484-afa0-3e572a46a04a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:27.316410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b4024-679e-4484-afa0-3e572a46a04a None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:27.318180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 58/114] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:27.329982 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea2140-ecfe-4402-87f2-68cf81510e5a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:27.330652 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea2140-ecfe-4402-87f2-68cf81510e5a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:27.331120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea2140-ecfe-4402-87f2-68cf81510e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:27.357709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-aaea2140-ecfe-4402-87f2-68cf81510e5a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:27.359246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-aaea2140-ecfe-4402-87f2-68cf81510e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:27.456704 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 57/115] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:27.584425 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f92c3d-144c-4f07-88cc-4765183c897c None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:27.585015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f92c3d-144c-4f07-88cc-4765183c897c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:27.585499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f92c3d-144c-4f07-88cc-4765183c897c None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:27.587283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 59/116] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:27.592246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98b534-9395-4170-b93c-167b893c89b2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:27.592780 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98b534-9395-4170-b93c-167b893c89b2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:27.593265 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98b534-9395-4170-b93c-167b893c89b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:27.603945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f92c3d-144c-4f07-88cc-4765183c897c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:27.626337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2d98b534-9395-4170-b93c-167b893c89b2 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:27.630240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2d98b534-9395-4170-b93c-167b893c89b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:27.706582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:27.779133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 58/117] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:27.779985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98b534-9395-4170-b93c-167b893c89b2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:27.787396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f92c3d-144c-4f07-88cc-4765183c897c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:27.829426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d8500-7798-4863-ad3b-4ed64c2760b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:27.829993 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d8500-7798-4863-ad3b-4ed64c2760b3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:27.830949 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d8500-7798-4863-ad3b-4ed64c2760b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:27.831296 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d8500-7798-4863-ad3b-4ed64c2760b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:27.851780 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d8500-7798-4863-ad3b-4ed64c2760b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:27.859771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-d53d8500-7798-4863-ad3b-4ed64c2760b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:27.859771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:27.859771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:27.859771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:27.859771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:27.859771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:27.859771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:27.859771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:27.859771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:27.859771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:27.859771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:27.859771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:27.859771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:27.859771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:27.859771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:27.859771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:27.860753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:27.860753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:27.860753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:27.860753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:27.860753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:27.860753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:27.860753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:27.860753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:27.860753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:27.860753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:27.860753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:27.860753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:27.860753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:27.860753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:27.860753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:27.860753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:27.862106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:27.862106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:27.862106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:27.862106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:27.862106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:27.862106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:27.862106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:27.862106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:27.862106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:27.862106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:27.862106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:27.862106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:27.865050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 60/118] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:41:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:41:27.866070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d8500-7798-4863-ad3b-4ed64c2760b3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:27.906592 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:27.909660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98b534-9395-4170-b93c-167b893c89b2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:27.946477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad358a-bce5-496c-8024-778f2160b278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:27.947126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad358a-bce5-496c-8024-778f2160b278 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:27.947409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad358a-bce5-496c-8024-778f2160b278 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:27.947684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad358a-bce5-496c-8024-778f2160b278 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:27.964627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad358a-bce5-496c-8024-778f2160b278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:27.970129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-17ad358a-bce5-496c-8024-778f2160b278 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:41:27.970129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:27.970129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:27.970129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:27.970129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:27.970129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:27.970129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:27.970129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:27.970129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:27.970129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:27.970129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:27.970129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:27.970129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:27.970129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:27.970129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:27.970129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:27.971569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:27.971569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:27.971569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:27.971569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:27.971569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:27.971569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:27.971569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:27.971569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:27.971569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:27.971569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:27.971569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:27.971569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:27.971569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:27.971569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:27.971569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:27.971569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:27.972748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:27.972748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:27.972748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:27.972748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:27.972748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:27.972748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:27.972748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:27.972748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:27.972748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:27.972748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:27.972748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:41:27.972748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:27.972748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 59/119] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:41:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:27.973379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad358a-bce5-496c-8024-778f2160b278 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:28.032268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d8500-7798-4863-ad3b-4ed64c2760b3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:28.065002 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:28.101170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d30b21-1a7d-4965-9c9b-f9295b80647f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:28.101812 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d30b21-1a7d-4965-9c9b-f9295b80647f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:28.102092 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d30b21-1a7d-4965-9c9b-f9295b80647f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:28.102365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d30b21-1a7d-4965-9c9b-f9295b80647f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:28.122926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d30b21-1a7d-4965-9c9b-f9295b80647f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:28.129135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad358a-bce5-496c-8024-778f2160b278 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:28.129608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-68d30b21-1a7d-4965-9c9b-f9295b80647f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:28.129608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:28.129608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:28.129608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:28.129608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:28.129608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:28.129608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:28.129608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:28.129608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:28.129608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:28.129608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:28.129608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:28.129608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:28.129608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:28.129608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:28.129608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:28.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:28.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:28.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:28.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:28.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:28.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:28.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:28.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:28.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:28.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:28.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:28.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:28.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:28.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:28.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:28.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:28.131133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:28.131133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:28.131133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:28.131133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:28.131133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:28.131133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:28.131133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:28.131133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:28.131133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:28.131133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:28.131133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:28.131133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:28.131133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 61/120] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:28.132714 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d30b21-1a7d-4965-9c9b-f9295b80647f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:28.165523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565941e-b437-446d-a0d1-86f1100c269b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:28.166199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565941e-b437-446d-a0d1-86f1100c269b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:28.166579 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565941e-b437-446d-a0d1-86f1100c269b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:28.166920 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565941e-b437-446d-a0d1-86f1100c269b None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:28.167656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565941e-b437-446d-a0d1-86f1100c269b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:28.168034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565941e-b437-446d-a0d1-86f1100c269b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:28.169544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565941e-b437-446d-a0d1-86f1100c269b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:28.178648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 60/121] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:28.179374 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565941e-b437-446d-a0d1-86f1100c269b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:28.256134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:28.267487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d30b21-1a7d-4965-9c9b-f9295b80647f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:28.279175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:28.287705 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565941e-b437-446d-a0d1-86f1100c269b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:28.313284 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d8f11-e11e-466e-a366-f1243aea0a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:28.315227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d8f11-e11e-466e-a366-f1243aea0a66 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:28.315732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d8f11-e11e-466e-a366-f1243aea0a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:28.316218 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d8f11-e11e-466e-a366-f1243aea0a66 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:28.317577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d8f11-e11e-466e-a366-f1243aea0a66 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:28.318064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d8f11-e11e-466e-a366-f1243aea0a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:28.320304 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d8f11-e11e-466e-a366-f1243aea0a66 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:28.325781 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d6d57-26bc-4460-887a-93c94c1541e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:28.326502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d6d57-26bc-4460-887a-93c94c1541e1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:28.326854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d6d57-26bc-4460-887a-93c94c1541e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:28.327201 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d6d57-26bc-4460-887a-93c94c1541e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:28.333727 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 62/122] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:41:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:28.338477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d8f11-e11e-466e-a366-f1243aea0a66 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:28.342488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d6d57-26bc-4460-887a-93c94c1541e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:28.348432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-7a4d6d57-26bc-4460-887a-93c94c1541e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:28.348432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:28.348432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:28.348432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:28.348432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:28.348432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:28.348432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:28.348432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:28.348432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:28.348432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:28.348432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:28.348432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:28.348432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:28.348432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:28.348432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:28.348432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:28.349319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:28.349319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:28.349319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:28.349319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:28.349319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:28.349319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:28.349319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:28.349319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:28.349319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:28.349319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:28.349319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:28.349319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:28.349319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:28.349319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:28.349319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:28.349319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:28.350077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:28.350077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:28.350077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:28.350077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:28.350077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:28.350077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:28.350077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:28.350077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:28.350077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:28.350077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:28.350077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:28.350077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:28.350077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 61/123] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:28.350697 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d6d57-26bc-4460-887a-93c94c1541e1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:28.473452 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d8f11-e11e-466e-a366-f1243aea0a66 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:28.495896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d6d57-26bc-4460-887a-93c94c1541e1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:28.522852 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6190f03-2736-4477-a319-e95656da0f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:28.524168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6190f03-2736-4477-a319-e95656da0f4e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:28.524553 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6190f03-2736-4477-a319-e95656da0f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:28.525212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6190f03-2736-4477-a319-e95656da0f4e None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:28.525975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6190f03-2736-4477-a319-e95656da0f4e None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:28.526593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6190f03-2736-4477-a319-e95656da0f4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:28.528797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6190f03-2736-4477-a319-e95656da0f4e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:28.539700 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ed36c-46df-4b58-970f-24ee5c8e6dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:28.539857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 63/124] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:28.540185 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ed36c-46df-4b58-970f-24ee5c8e6dac None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:28.540477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ed36c-46df-4b58-970f-24ee5c8e6dac None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:28.540855 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ed36c-46df-4b58-970f-24ee5c8e6dac None admin] PATH_INFO: `/v3/users/984123d0eaef4929af01f39a46ebd51e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:28.542062 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ed36c-46df-4b58-970f-24ee5c8e6dac None admin] RBAC: Authorizing `identity:get_user(user_id=984123d0eaef4929af01f39a46ebd51e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:28.543513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ed36c-46df-4b58-970f-24ee5c8e6dac None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:28.545060 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6190f03-2736-4477-a319-e95656da0f4e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:28.545500 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 62/125] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:28 2022] GET /identity/v3/users/984123d0eaef4929af01f39a46ebd51e => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:28.546631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ed36c-46df-4b58-970f-24ee5c8e6dac None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:28.566328 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:28.582224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:28.666269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6190f03-2736-4477-a319-e95656da0f4e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:28.686430 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ed36c-46df-4b58-970f-24ee5c8e6dac None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:28.709538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd9455-4a87-4be5-a3a0-e1d03f7a1300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:28.710370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd9455-4a87-4be5-a3a0-e1d03f7a1300 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:28.710744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd9455-4a87-4be5-a3a0-e1d03f7a1300 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:28.711116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd9455-4a87-4be5-a3a0-e1d03f7a1300 None admin] PATH_INFO: `/v3/users/3d9981f1b2c1421b8d5a89485cc87def` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:28.712390 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd9455-4a87-4be5-a3a0-e1d03f7a1300 None admin] RBAC: Authorizing `identity:get_user(user_id=3d9981f1b2c1421b8d5a89485cc87def)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:28.714721 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd9455-4a87-4be5-a3a0-e1d03f7a1300 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:28.717085 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 64/126] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:28 2022] GET /identity/v3/users/3d9981f1b2c1421b8d5a89485cc87def => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:28.718393 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd9455-4a87-4be5-a3a0-e1d03f7a1300 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:28.729263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b77b6c-7f44-4ba7-b729-c7c63ae1f43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:28.730072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b77b6c-7f44-4ba7-b729-c7c63ae1f43b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:28.730585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b77b6c-7f44-4ba7-b729-c7c63ae1f43b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:28.731056 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b77b6c-7f44-4ba7-b729-c7c63ae1f43b None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:28.731968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b77b6c-7f44-4ba7-b729-c7c63ae1f43b None admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:28.732506 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b77b6c-7f44-4ba7-b729-c7c63ae1f43b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:28.734068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b77b6c-7f44-4ba7-b729-c7c63ae1f43b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:28.743182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 63/127] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:28.748958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b77b6c-7f44-4ba7-b729-c7c63ae1f43b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:28.812010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:28.836218 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd9455-4a87-4be5-a3a0-e1d03f7a1300 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:28.866489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:28.872553 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab99016-0afb-43d2-9ca4-49f4db1b3e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:28.873200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab99016-0afb-43d2-9ca4-49f4db1b3e32 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:28.873485 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab99016-0afb-43d2-9ca4-49f4db1b3e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:28.873763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab99016-0afb-43d2-9ca4-49f4db1b3e32 None admin] PATH_INFO: `/v3/users/3d9981f1b2c1421b8d5a89485cc87def` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:28.874909 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab99016-0afb-43d2-9ca4-49f4db1b3e32 None admin] RBAC: Authorizing `identity:get_user(user_id=3d9981f1b2c1421b8d5a89485cc87def)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:28.876433 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab99016-0afb-43d2-9ca4-49f4db1b3e32 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:28.878760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 65/128] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:28 2022] GET /identity/v3/users/3d9981f1b2c1421b8d5a89485cc87def => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:28.879363 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab99016-0afb-43d2-9ca4-49f4db1b3e32 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:28.889403 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b77b6c-7f44-4ba7-b729-c7c63ae1f43b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:28.936512 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763aea1-d329-4f1b-8da0-7c56a534f9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:28.937752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763aea1-d329-4f1b-8da0-7c56a534f9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:28.938283 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763aea1-d329-4f1b-8da0-7c56a534f9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:28.939123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763aea1-d329-4f1b-8da0-7c56a534f9d2 None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:28.942680 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763aea1-d329-4f1b-8da0-7c56a534f9d2 None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:28.944888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763aea1-d329-4f1b-8da0-7c56a534f9d2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:28.948067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 64/129] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:28 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:28.949152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763aea1-d329-4f1b-8da0-7c56a534f9d2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:28.987435 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab99016-0afb-43d2-9ca4-49f4db1b3e32 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:29.031707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4373f5-2a2d-4e39-8e66-826b70e6e196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:29.032871 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4373f5-2a2d-4e39-8e66-826b70e6e196 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:29.034284 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4373f5-2a2d-4e39-8e66-826b70e6e196 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:29.034762 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4373f5-2a2d-4e39-8e66-826b70e6e196 None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:29.036351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4373f5-2a2d-4e39-8e66-826b70e6e196 None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:29.038853 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4373f5-2a2d-4e39-8e66-826b70e6e196 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:29.041937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 66/130] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:28 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:29.042885 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4373f5-2a2d-4e39-8e66-826b70e6e196 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:29.067085 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:29.084691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763aea1-d329-4f1b-8da0-7c56a534f9d2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:29.128368 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de49c1-a0d3-42e6-b2b2-88d414e0b01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:29.129169 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de49c1-a0d3-42e6-b2b2-88d414e0b01a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:29.129698 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de49c1-a0d3-42e6-b2b2-88d414e0b01a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:29.129880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de49c1-a0d3-42e6-b2b2-88d414e0b01a None admin] PATH_INFO: `/v3/users/3d9981f1b2c1421b8d5a89485cc87def` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:29.131162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de49c1-a0d3-42e6-b2b2-88d414e0b01a None admin] RBAC: Authorizing `identity:get_user(user_id=3d9981f1b2c1421b8d5a89485cc87def)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:29.133221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de49c1-a0d3-42e6-b2b2-88d414e0b01a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:29.138972 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 65/131] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:28 2022] GET /identity/v3/users/3d9981f1b2c1421b8d5a89485cc87def => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:29.138972 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de49c1-a0d3-42e6-b2b2-88d414e0b01a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:29.159685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:29.180284 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4373f5-2a2d-4e39-8e66-826b70e6e196 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:29.216971 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7fafc-0f7b-45e9-84a3-bf378cfdfc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:29.217768 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7fafc-0f7b-45e9-84a3-bf378cfdfc18 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:29.218167 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7fafc-0f7b-45e9-84a3-bf378cfdfc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:29.218539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7fafc-0f7b-45e9-84a3-bf378cfdfc18 None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:29.219841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7fafc-0f7b-45e9-84a3-bf378cfdfc18 None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:29.221545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7fafc-0f7b-45e9-84a3-bf378cfdfc18 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:29.223683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 67/132] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:29 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:29.224914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7fafc-0f7b-45e9-84a3-bf378cfdfc18 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:29.270767 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de49c1-a0d3-42e6-b2b2-88d414e0b01a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:29.320228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf67d6-c2b0-4a23-9327-5f939514966f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:29.320427 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:29.320893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf67d6-c2b0-4a23-9327-5f939514966f None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:29.321470 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf67d6-c2b0-4a23-9327-5f939514966f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:29.321470 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf67d6-c2b0-4a23-9327-5f939514966f None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5/users/984123d0eaef4929af01f39a46ebd51e/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:29.326573 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf67d6-c2b0-4a23-9327-5f939514966f None admin] RBAC: Authorizing `identity:create_grant(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5, user_id=984123d0eaef4929af01f39a46ebd51e, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:29.328848 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf67d6-c2b0-4a23-9327-5f939514966f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:29.338898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7fafc-0f7b-45e9-84a3-bf378cfdfc18 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:29.349718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 66/133] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:41:29 2022] PUT /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5/users/984123d0eaef4929af01f39a46ebd51e/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:41:29.350888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf67d6-c2b0-4a23-9327-5f939514966f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:29.458202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef335bd-e7b5-4552-bce9-963bca130d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:29.458891 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef335bd-e7b5-4552-bce9-963bca130d92 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:29.459176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef335bd-e7b5-4552-bce9-963bca130d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:29.459453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef335bd-e7b5-4552-bce9-963bca130d92 None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5/users/3d9981f1b2c1421b8d5a89485cc87def/roles/84a30faea3e94eb48b03ec7ceb7c1392` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:29.464773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef335bd-e7b5-4552-bce9-963bca130d92 None admin] RBAC: Authorizing `identity:create_grant(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5, user_id=3d9981f1b2c1421b8d5a89485cc87def, role_id=84a30faea3e94eb48b03ec7ceb7c1392)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:29.465380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:29.466316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef335bd-e7b5-4552-bce9-963bca130d92 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:29.489227 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf67d6-c2b0-4a23-9327-5f939514966f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:29.530733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 68/134] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:41:29 2022] PUT /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5/users/3d9981f1b2c1421b8d5a89485cc87def/roles/84a30faea3e94eb48b03ec7ceb7c1392 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:41:29.531520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef335bd-e7b5-4552-bce9-963bca130d92 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:29.535766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415827b-551d-426d-9f45-b5896266dfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:29.536525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415827b-551d-426d-9f45-b5896266dfba None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:29.536814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415827b-551d-426d-9f45-b5896266dfba None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:29.537093 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415827b-551d-426d-9f45-b5896266dfba None admin] PATH_INFO: `/v3/projects/bfab41741dcd4147ae89996789a3dbaa` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:29.540993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415827b-551d-426d-9f45-b5896266dfba None admin] RBAC: Authorizing `identity:get_project(project_id=bfab41741dcd4147ae89996789a3dbaa)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:29.542549 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415827b-551d-426d-9f45-b5896266dfba None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:29.544819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 67/135] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:29 2022] GET /identity/v3/projects/bfab41741dcd4147ae89996789a3dbaa => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:29.551026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415827b-551d-426d-9f45-b5896266dfba None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:29.720171 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef335bd-e7b5-4552-bce9-963bca130d92 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:29.793414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a463f55-e429-4e8e-a6b3-e78ace395f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:29.794484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a463f55-e429-4e8e-a6b3-e78ace395f84 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:29.794974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a463f55-e429-4e8e-a6b3-e78ace395f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:29.795462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a463f55-e429-4e8e-a6b3-e78ace395f84 None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5/users/3d9981f1b2c1421b8d5a89485cc87def/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:29.797955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a463f55-e429-4e8e-a6b3-e78ace395f84 None admin] RBAC: Authorizing `identity:create_grant(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5, user_id=3d9981f1b2c1421b8d5a89485cc87def, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:29.800374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a463f55-e429-4e8e-a6b3-e78ace395f84 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:29.820057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 69/136] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:41:29 2022] PUT /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5/users/3d9981f1b2c1421b8d5a89485cc87def/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:41:29.822070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816e737-ff7a-442a-aaf2-acbe12c13b72 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:29.822577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816e737-ff7a-442a-aaf2-acbe12c13b72 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:29.823130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415827b-551d-426d-9f45-b5896266dfba None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:29.823492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816e737-ff7a-442a-aaf2-acbe12c13b72 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:29.825595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 70/137] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:29.833610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d2f42-c4c2-4ae7-9ac0-180d32e64f7c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:29.834281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d2f42-c4c2-4ae7-9ac0-180d32e64f7c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:29.834784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d2f42-c4c2-4ae7-9ac0-180d32e64f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:29.872482 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:29.877889 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-4c7d2f42-c4c2-4ae7-9ac0-180d32e64f7c None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:29.879367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599aff5-3775-4d7f-a9e2-df69ea487684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:29.881172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599aff5-3775-4d7f-a9e2-df69ea487684 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:29.881172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599aff5-3775-4d7f-a9e2-df69ea487684 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:29.881172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599aff5-3775-4d7f-a9e2-df69ea487684 None admin] PATH_INFO: `/v3/projects/bfab41741dcd4147ae89996789a3dbaa/users/3d9981f1b2c1421b8d5a89485cc87def/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:29.885473 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599aff5-3775-4d7f-a9e2-df69ea487684 None admin] RBAC: Authorizing `identity:create_grant(project_id=bfab41741dcd4147ae89996789a3dbaa, user_id=3d9981f1b2c1421b8d5a89485cc87def, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:29.885619 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4c7d2f42-c4c2-4ae7-9ac0-180d32e64f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:29.887064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599aff5-3775-4d7f-a9e2-df69ea487684 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:29.923601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 68/138] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:41:29 2022] PUT /identity/v3/projects/bfab41741dcd4147ae89996789a3dbaa/users/3d9981f1b2c1421b8d5a89485cc87def/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:41:29.961656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303ca96-7f87-46c2-904e-602bc26e2c28 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:29.961963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303ca96-7f87-46c2-904e-602bc26e2c28 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:29.962226 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303ca96-7f87-46c2-904e-602bc26e2c28 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:29.963753 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 69/139] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:29.971616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78324c6-b808-4bae-af2a-be14a73368ea None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:29.972383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78324c6-b808-4bae-af2a-be14a73368ea None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:29.972858 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78324c6-b808-4bae-af2a-be14a73368ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:30.006302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 71/140] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:30.013278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e78324c6-b808-4bae-af2a-be14a73368ea None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:30.014837 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e78324c6-b808-4bae-af2a-be14a73368ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:30.017830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:30.118655 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 70/141] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:30.424496 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d2f42-c4c2-4ae7-9ac0-180d32e64f7c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:30.550884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78324c6-b808-4bae-af2a-be14a73368ea None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:30.574646 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d2f42-c4c2-4ae7-9ac0-180d32e64f7c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:30.652958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa169d57-9ec3-4feb-8795-708e65fda701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:30.653927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa169d57-9ec3-4feb-8795-708e65fda701 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:30.654405 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa169d57-9ec3-4feb-8795-708e65fda701 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:30.654878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa169d57-9ec3-4feb-8795-708e65fda701 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:30.665483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:30.692723 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa169d57-9ec3-4feb-8795-708e65fda701 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:30.698116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-aa169d57-9ec3-4feb-8795-708e65fda701 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:41:30.698116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:30.698116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:30.698116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:30.698116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:30.698116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:30.698116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:30.698116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:30.698116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:30.698116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:30.698116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:30.698116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:30.698116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:30.698116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:30.698116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:30.698116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:30.698944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:30.698944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:30.698944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:30.698944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:30.698944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:30.698944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:30.698944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:30.698944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:30.698944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:30.698944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:30.698944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:30.698944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:30.698944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:30.698944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:30.698944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:30.698944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:30.699900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:30.699900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:30.699900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:30.699900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:30.699900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:30.699900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:30.699900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:30.699900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:30.699900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:30.699900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:30.699900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:41:30.699900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:30.699900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 72/142] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:41:30.702928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78324c6-b808-4bae-af2a-be14a73368ea None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:30.705689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa169d57-9ec3-4feb-8795-708e65fda701 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:30.743175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54893483-dbbb-4608-ada1-63d3fbe1709f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:30.744049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54893483-dbbb-4608-ada1-63d3fbe1709f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:30.744633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54893483-dbbb-4608-ada1-63d3fbe1709f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:30.745064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54893483-dbbb-4608-ada1-63d3fbe1709f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:30.746562 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54893483-dbbb-4608-ada1-63d3fbe1709f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:30.748418 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54893483-dbbb-4608-ada1-63d3fbe1709f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:30.750535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 71/143] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:30.760015 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54893483-dbbb-4608-ada1-63d3fbe1709f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:30.784539 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:30.871117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54893483-dbbb-4608-ada1-63d3fbe1709f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:30.910008 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17741d23-037a-44e0-9ae6-e9643935217f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:30.910831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17741d23-037a-44e0-9ae6-e9643935217f None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:30.911251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17741d23-037a-44e0-9ae6-e9643935217f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:30.911938 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17741d23-037a-44e0-9ae6-e9643935217f None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:30.912748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17741d23-037a-44e0-9ae6-e9643935217f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:30.914959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17741d23-037a-44e0-9ae6-e9643935217f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:30.949458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 72/144] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:41:30 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:41:31.000234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa169d57-9ec3-4feb-8795-708e65fda701 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:31.051341 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b422e8-a179-4eee-b41c-ee44b8a308bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:31.051955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b422e8-a179-4eee-b41c-ee44b8a308bf None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:31.052257 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b422e8-a179-4eee-b41c-ee44b8a308bf None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:31.052528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b422e8-a179-4eee-b41c-ee44b8a308bf None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:31.053192 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b422e8-a179-4eee-b41c-ee44b8a308bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:31.053493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b422e8-a179-4eee-b41c-ee44b8a308bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:31.054867 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b422e8-a179-4eee-b41c-ee44b8a308bf None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:31.068927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 73/145] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:31.074534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17741d23-037a-44e0-9ae6-e9643935217f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:31.164886 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:31.208125 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17741d23-037a-44e0-9ae6-e9643935217f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:31.245048 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a724f0-64fc-4f4c-b771-064d85e6ac88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:31.245731 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a724f0-64fc-4f4c-b771-064d85e6ac88 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:31.246011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a724f0-64fc-4f4c-b771-064d85e6ac88 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:31.246281 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a724f0-64fc-4f4c-b771-064d85e6ac88 None admin] PATH_INFO: `/v3/users/a9894cf5ac414bcfa637feb6d3fd1943` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:31.247429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a724f0-64fc-4f4c-b771-064d85e6ac88 None admin] RBAC: Authorizing `identity:get_user(user_id=a9894cf5ac414bcfa637feb6d3fd1943)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:31.248911 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a724f0-64fc-4f4c-b771-064d85e6ac88 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:31.250656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 73/146] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:31 2022] GET /identity/v3/users/a9894cf5ac414bcfa637feb6d3fd1943 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:31.257208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b422e8-a179-4eee-b41c-ee44b8a308bf None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:31.320428 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:31.365153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b422e8-a179-4eee-b41c-ee44b8a308bf None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:31.400473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0ba40-f028-4819-8cce-4397c9db46db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:31.401138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0ba40-f028-4819-8cce-4397c9db46db None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:31.401416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0ba40-f028-4819-8cce-4397c9db46db None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:31.401685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0ba40-f028-4819-8cce-4397c9db46db None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:31.402808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0ba40-f028-4819-8cce-4397c9db46db None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:31.404294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0ba40-f028-4819-8cce-4397c9db46db None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:31.406083 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 74/147] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:31 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:31.412493 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a724f0-64fc-4f4c-b771-064d85e6ac88 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:31.515031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a724f0-64fc-4f4c-b771-064d85e6ac88 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:31.551198 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d04429-a439-4258-a997-2c64edfac7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:31.551913 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d04429-a439-4258-a997-2c64edfac7af None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:31.552153 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d04429-a439-4258-a997-2c64edfac7af None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:31.552436 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d04429-a439-4258-a997-2c64edfac7af None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5/users/a9894cf5ac414bcfa637feb6d3fd1943/roles/a5350d487c374ea680a13ac3044168fc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:31.554380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d04429-a439-4258-a997-2c64edfac7af None admin] RBAC: Authorizing `identity:create_grant(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5, user_id=a9894cf5ac414bcfa637feb6d3fd1943, role_id=a5350d487c374ea680a13ac3044168fc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:31.555915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d04429-a439-4258-a997-2c64edfac7af None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:31.564345 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:31.579650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 74/148] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:41:31 2022] PUT /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5/users/a9894cf5ac414bcfa637feb6d3fd1943/roles/a5350d487c374ea680a13ac3044168fc => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:41:32.274276 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769a414-dfe2-4a6d-b65c-3c295a17e574 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:32.274874 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769a414-dfe2-4a6d-b65c-3c295a17e574 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:32.274874 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769a414-dfe2-4a6d-b65c-3c295a17e574 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:32.276451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 75/149] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:32.282101 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f5af2-8ca4-46e3-9078-9b5248c39cec None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:32.282374 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f5af2-8ca4-46e3-9078-9b5248c39cec None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:32.282646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f5af2-8ca4-46e3-9078-9b5248c39cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:32.302875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e36f5af2-8ca4-46e3-9078-9b5248c39cec None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:32.304132 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e36f5af2-8ca4-46e3-9078-9b5248c39cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:32.383297 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 75/150] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:32.861813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d018ffc-f03f-4a09-9b61-409101392423 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:32.862119 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d018ffc-f03f-4a09-9b61-409101392423 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:32.862350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d018ffc-f03f-4a09-9b61-409101392423 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:32.864011 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 76/151] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:32.870851 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1319d-8ce2-40fa-9f3c-3a8e0c4e449f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:32.871366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1319d-8ce2-40fa-9f3c-3a8e0c4e449f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:32.871366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1319d-8ce2-40fa-9f3c-3a8e0c4e449f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:32.892607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d9c1319d-8ce2-40fa-9f3c-3a8e0c4e449f None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:32.894225 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d9c1319d-8ce2-40fa-9f3c-3a8e0c4e449f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:32.935037 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d018ffc-f03f-4a09-9b61-409101392423 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:32.976426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 76/152] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:33.037730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b8dec-1f3e-49c3-b32d-4cfeb9ee1ad3 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:33.038353 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b8dec-1f3e-49c3-b32d-4cfeb9ee1ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:33.038824 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b8dec-1f3e-49c3-b32d-4cfeb9ee1ad3 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:33.040415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 77/153] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:41:33.045962 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297a60b-0fa5-46be-97ef-e24357ecf4d4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:33.046616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297a60b-0fa5-46be-97ef-e24357ecf4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:33.047212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297a60b-0fa5-46be-97ef-e24357ecf4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:33.073340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2297a60b-0fa5-46be-97ef-e24357ecf4d4 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:33.075051 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2297a60b-0fa5-46be-97ef-e24357ecf4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:33.115522 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d018ffc-f03f-4a09-9b61-409101392423 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:33.132513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:33.159746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 78/154] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:33.197920 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:33.212282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce3d3f-fa39-4fc3-961e-143f75ef8317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:33.212926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce3d3f-fa39-4fc3-961e-143f75ef8317 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:33.213207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce3d3f-fa39-4fc3-961e-143f75ef8317 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:33.213483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce3d3f-fa39-4fc3-961e-143f75ef8317 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:33.232510 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce3d3f-fa39-4fc3-961e-143f75ef8317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:33.238099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-70ce3d3f-fa39-4fc3-961e-143f75ef8317 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:33.238099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:33.238099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:33.238099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:33.238099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:33.238099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:33.238099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:33.238099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:33.238099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:33.238099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:33.238099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:33.238099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:33.238099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:33.238099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:33.238099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:33.238099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:33.238997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:33.238997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:33.238997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:33.238997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:33.238997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:33.238997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:33.238997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:33.238997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:33.238997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:33.238997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:33.238997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:33.238997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:33.238997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:33.238997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:33.238997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:33.238997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:33.239756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:33.239756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:33.239756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:33.239756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:33.239756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:33.239756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:33.239756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:33.239756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:33.239756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:33.239756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:33.239756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:33.239756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:33.239756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 77/155] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:41:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:41:33.246711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297a60b-0fa5-46be-97ef-e24357ecf4d4 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:33.388103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297a60b-0fa5-46be-97ef-e24357ecf4d4 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:33.435273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2d28e-a7d0-4aa0-bc78-7a9c45dc791a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:33.435885 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2d28e-a7d0-4aa0-bc78-7a9c45dc791a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:33.436187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2d28e-a7d0-4aa0-bc78-7a9c45dc791a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:33.436463 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2d28e-a7d0-4aa0-bc78-7a9c45dc791a None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:33.437134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2d28e-a7d0-4aa0-bc78-7a9c45dc791a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:33.437495 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2d28e-a7d0-4aa0-bc78-7a9c45dc791a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:33.438980 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2d28e-a7d0-4aa0-bc78-7a9c45dc791a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:33.450303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 79/156] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:33.456512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce3d3f-fa39-4fc3-961e-143f75ef8317 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:33.515200 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2d28e-a7d0-4aa0-bc78-7a9c45dc791a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:33.595616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:33.630101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce3d3f-fa39-4fc3-961e-143f75ef8317 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:33.695840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb47e6e-d749-4fdc-a88b-fbaffa839d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:33.696423 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb47e6e-d749-4fdc-a88b-fbaffa839d66 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:33.696987 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb47e6e-d749-4fdc-a88b-fbaffa839d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:33.697256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb47e6e-d749-4fdc-a88b-fbaffa839d66 None admin] PATH_INFO: `/v3/users/984123d0eaef4929af01f39a46ebd51e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:33.700137 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb47e6e-d749-4fdc-a88b-fbaffa839d66 None admin] RBAC: Authorizing `identity:get_user(user_id=984123d0eaef4929af01f39a46ebd51e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:33.701888 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb47e6e-d749-4fdc-a88b-fbaffa839d66 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:33.707860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 78/157] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:33 2022] GET /identity/v3/users/984123d0eaef4929af01f39a46ebd51e => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:33.716729 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb47e6e-d749-4fdc-a88b-fbaffa839d66 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:33.728340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2d28e-a7d0-4aa0-bc78-7a9c45dc791a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:33.816801 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a44c0-3c22-4a5e-badf-bc6d96581275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:33.817803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a44c0-3c22-4a5e-badf-bc6d96581275 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:33.818315 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a44c0-3c22-4a5e-badf-bc6d96581275 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:33.819055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a44c0-3c22-4a5e-badf-bc6d96581275 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:33.834899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:33.840278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a44c0-3c22-4a5e-badf-bc6d96581275 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:33.846256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d70a44c0-3c22-4a5e-badf-bc6d96581275 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:33.846256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:33.846256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:33.846256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:33.846256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:33.846256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:33.846256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:33.846256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:33.846256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:33.846256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:33.846256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:33.846256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:33.846256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:33.846256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:33.846256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:33.846256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:33.847242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:33.847242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:33.847242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:33.847242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:33.847242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:33.847242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:33.847242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:33.847242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:33.847242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:33.847242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:33.847242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:33.847242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:33.847242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:33.847242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:33.847242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:33.847242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:33.847999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:33.847999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:33.847999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:33.847999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:33.847999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:33.847999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:33.847999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:33.847999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:33.847999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:33.847999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:33.847999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:33.847999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:33.856168 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 80/158] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:41:33.857290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a44c0-3c22-4a5e-badf-bc6d96581275 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:33.926007 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb47e6e-d749-4fdc-a88b-fbaffa839d66 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:34.000227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c1810-42e1-4d47-a5b8-37d104adb500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:34.000763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c1810-42e1-4d47-a5b8-37d104adb500 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:34.001035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c1810-42e1-4d47-a5b8-37d104adb500 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:34.001612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c1810-42e1-4d47-a5b8-37d104adb500 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:34.045131 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a44c0-3c22-4a5e-badf-bc6d96581275 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:34.051779 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:34.118303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45023b15-0f53-4a5d-9ba8-d0a0a4318c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:34.118918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45023b15-0f53-4a5d-9ba8-d0a0a4318c85 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:34.119199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45023b15-0f53-4a5d-9ba8-d0a0a4318c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:34.119473 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45023b15-0f53-4a5d-9ba8-d0a0a4318c85 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:34.120162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45023b15-0f53-4a5d-9ba8-d0a0a4318c85 None admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:34.120466 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45023b15-0f53-4a5d-9ba8-d0a0a4318c85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:34.121865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45023b15-0f53-4a5d-9ba8-d0a0a4318c85 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:34.137771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c1810-42e1-4d47-a5b8-37d104adb500 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:34.139258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c1810-42e1-4d47-a5b8-37d104adb500 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:34.142733 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 81/159] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:34.144501 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45023b15-0f53-4a5d-9ba8-d0a0a4318c85 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:34.238285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:34.336021 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 79/160] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 07:41:34.337318 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c098ea-b9ee-4ba0-8e2c-9bea0c38d3f2 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:34.337597 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c098ea-b9ee-4ba0-8e2c-9bea0c38d3f2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:34.337868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c098ea-b9ee-4ba0-8e2c-9bea0c38d3f2 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:34.344106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 80/161] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:34.344106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c098ea-b9ee-4ba0-8e2c-9bea0c38d3f2 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:34.369611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45023b15-0f53-4a5d-9ba8-d0a0a4318c85 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:34.431637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b567793-4fb8-46f1-99e4-123eb15a875a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:34.432729 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b567793-4fb8-46f1-99e4-123eb15a875a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:34.433229 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b567793-4fb8-46f1-99e4-123eb15a875a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:34.433707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b567793-4fb8-46f1-99e4-123eb15a875a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:34.449595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b567793-4fb8-46f1-99e4-123eb15a875a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:34.455955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-8b567793-4fb8-46f1-99e4-123eb15a875a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:41:34.455955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:34.455955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:34.455955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:34.455955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:34.455955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:34.455955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:34.455955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:34.455955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:34.455955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:34.455955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:34.455955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:34.455955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:34.455955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:34.455955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:34.455955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:34.456844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:34.456844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:34.456844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:34.456844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:34.456844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:34.456844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:34.456844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:34.456844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:34.456844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:34.456844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:34.456844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:34.456844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:34.456844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:34.456844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:34.456844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:34.456844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:34.457633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:34.457633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:34.457633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:34.457633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:34.457633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:34.457633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:34.457633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:34.457633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:34.457633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:34.457633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:34.457633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:41:34.457633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:34.460154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 82/162] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:41:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:34.461803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af68b80-82e5-4c94-aac8-ebd2dc199fdc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:34.462300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af68b80-82e5-4c94-aac8-ebd2dc199fdc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:34.462777 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af68b80-82e5-4c94-aac8-ebd2dc199fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:34.482413 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7af68b80-82e5-4c94-aac8-ebd2dc199fdc None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:34.483919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7af68b80-82e5-4c94-aac8-ebd2dc199fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:34.536512 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:34.558220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 83/163] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:34.559922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af68b80-82e5-4c94-aac8-ebd2dc199fdc None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:34.670873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af68b80-82e5-4c94-aac8-ebd2dc199fdc None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:34.705989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c098ea-b9ee-4ba0-8e2c-9bea0c38d3f2 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:34.711094 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca2e97-6348-42e8-b6dd-7e10aae29fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:34.711982 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca2e97-6348-42e8-b6dd-7e10aae29fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:34.712490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca2e97-6348-42e8-b6dd-7e10aae29fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:34.712951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca2e97-6348-42e8-b6dd-7e10aae29fc3 None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:34.714289 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca2e97-6348-42e8-b6dd-7e10aae29fc3 None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:34.715934 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca2e97-6348-42e8-b6dd-7e10aae29fc3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:34.718300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 84/164] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:34 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:34.719287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca2e97-6348-42e8-b6dd-7e10aae29fc3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:34.748454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:34.801362 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115b423-92c1-43fc-b8db-708e72e6aed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:34.802007 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115b423-92c1-43fc-b8db-708e72e6aed1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:34.802640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115b423-92c1-43fc-b8db-708e72e6aed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:34.802640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115b423-92c1-43fc-b8db-708e72e6aed1 None admin] PATH_INFO: `/v3/users/3d9981f1b2c1421b8d5a89485cc87def` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:34.803704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1115b423-92c1-43fc-b8db-708e72e6aed1 None admin] RBAC: Authorizing `identity:get_user(user_id=3d9981f1b2c1421b8d5a89485cc87def)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:34.805287 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1115b423-92c1-43fc-b8db-708e72e6aed1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:34.806868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 81/165] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:34 2022] GET /identity/v3/users/3d9981f1b2c1421b8d5a89485cc87def => generated 275 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:34.812140 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115b423-92c1-43fc-b8db-708e72e6aed1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:34.836848 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca2e97-6348-42e8-b6dd-7e10aae29fc3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:34.875776 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:34.876749 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0373ce-7ceb-486f-9b53-4c8c3aef3a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:34.878729 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0373ce-7ceb-486f-9b53-4c8c3aef3a40 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:34.879224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0373ce-7ceb-486f-9b53-4c8c3aef3a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:34.879691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0373ce-7ceb-486f-9b53-4c8c3aef3a40 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:34.880727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0373ce-7ceb-486f-9b53-4c8c3aef3a40 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:34.881246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0373ce-7ceb-486f-9b53-4c8c3aef3a40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:34.882953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0373ce-7ceb-486f-9b53-4c8c3aef3a40 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:34.892857 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 85/166] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:41:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:34.894275 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0373ce-7ceb-486f-9b53-4c8c3aef3a40 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:35.011585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:35.038863 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0373ce-7ceb-486f-9b53-4c8c3aef3a40 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:35.082675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115b423-92c1-43fc-b8db-708e72e6aed1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:35.106118 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac263f65-0f00-4c85-923c-761f5cce17c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:35.106793 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac263f65-0f00-4c85-923c-761f5cce17c5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:35.107018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac263f65-0f00-4c85-923c-761f5cce17c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:35.107294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac263f65-0f00-4c85-923c-761f5cce17c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:35.176805 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f2621-8808-42f9-9700-dcfacf6508d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:35.177429 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f2621-8808-42f9-9700-dcfacf6508d8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:35.177703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f2621-8808-42f9-9700-dcfacf6508d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:35.177967 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f2621-8808-42f9-9700-dcfacf6508d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:35.178691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f2621-8808-42f9-9700-dcfacf6508d8 None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5, user.id=984123d0eaef4929af01f39a46ebd51e) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:35.178991 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f2621-8808-42f9-9700-dcfacf6508d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:35.180444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f2621-8808-42f9-9700-dcfacf6508d8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:35.196882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 82/167] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:34 2022] GET /identity/v3/role_assignments?user.id=984123d0eaef4929af01f39a46ebd51e&scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 608 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:35.197829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f2621-8808-42f9-9700-dcfacf6508d8 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:35.237625 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac263f65-0f00-4c85-923c-761f5cce17c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:35.239145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac263f65-0f00-4c85-923c-761f5cce17c5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:35.404935 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:35.408583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:35.422198 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 86/168] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:41:35.422965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac263f65-0f00-4c85-923c-761f5cce17c5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:35.442869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f2621-8808-42f9-9700-dcfacf6508d8 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:35.581087 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac263f65-0f00-4c85-923c-761f5cce17c5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:35.595789 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268ffb5-1c02-4ad4-8af7-e8b739512c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:35.596505 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268ffb5-1c02-4ad4-8af7-e8b739512c6e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:35.596802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268ffb5-1c02-4ad4-8af7-e8b739512c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:35.597026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268ffb5-1c02-4ad4-8af7-e8b739512c6e None admin] PATH_INFO: `/v3/users/3d9981f1b2c1421b8d5a89485cc87def` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:35.600868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268ffb5-1c02-4ad4-8af7-e8b739512c6e None admin] RBAC: Authorizing `identity:get_user(user_id=3d9981f1b2c1421b8d5a89485cc87def)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:35.602366 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268ffb5-1c02-4ad4-8af7-e8b739512c6e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:35.604105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 83/169] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:35 2022] GET /identity/v3/users/3d9981f1b2c1421b8d5a89485cc87def => generated 275 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:35.604908 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268ffb5-1c02-4ad4-8af7-e8b739512c6e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:35.664950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa9e74-99c7-4458-b33b-f7ada46376eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:35.665937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa9e74-99c7-4458-b33b-f7ada46376eb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:35.666417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa9e74-99c7-4458-b33b-f7ada46376eb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:35.666906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa9e74-99c7-4458-b33b-f7ada46376eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:35.691484 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa9e74-99c7-4458-b33b-f7ada46376eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:35.704103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-47fa9e74-99c7-4458-b33b-f7ada46376eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:35.704103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:35.704103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:35.704103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:35.704103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:35.704103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:35.704103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:35.704103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:35.704103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:35.704103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:35.704103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:35.704103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:35.704103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:35.704103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:35.704103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:35.704103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:35.705078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:35.705078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:35.705078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:35.705078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:35.705078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:35.705078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:35.705078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:35.705078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:35.705078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:35.705078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:35.705078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:35.705078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:35.705078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:35.705078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:35.705078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:35.705078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:35.706310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:35.706310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:35.706310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:35.706310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:35.706310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:35.706310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:35.706310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:35.706310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:35.706310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:35.706310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:35.706310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:35.706310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:35.709012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 87/170] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:35.710087 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa9e74-99c7-4458-b33b-f7ada46376eb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:35.766834 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:35.828197 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268ffb5-1c02-4ad4-8af7-e8b739512c6e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:35.869042 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa9e74-99c7-4458-b33b-f7ada46376eb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:35.886239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbf3a6-295c-451c-a8b1-2ed51f674b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:35.886897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbf3a6-295c-451c-a8b1-2ed51f674b71 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:35.887158 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbf3a6-295c-451c-a8b1-2ed51f674b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:35.887431 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbf3a6-295c-451c-a8b1-2ed51f674b71 None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:35.888598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbf3a6-295c-451c-a8b1-2ed51f674b71 None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:35.890057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbf3a6-295c-451c-a8b1-2ed51f674b71 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:35.891916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 84/171] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:35 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:35.892603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbf3a6-295c-451c-a8b1-2ed51f674b71 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:35.911664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18098155-489f-4074-951e-fdee6597a283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:35.912653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18098155-489f-4074-951e-fdee6597a283 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:35.913184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18098155-489f-4074-951e-fdee6597a283 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:35.913675 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18098155-489f-4074-951e-fdee6597a283 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:35.940954 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:36.036425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18098155-489f-4074-951e-fdee6597a283 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:36.038154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18098155-489f-4074-951e-fdee6597a283 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:36.065689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbf3a6-295c-451c-a8b1-2ed51f674b71 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:36.097984 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:36.109555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07bcd7-8f7f-4071-9b7b-150b15f776c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:36.110573 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07bcd7-8f7f-4071-9b7b-150b15f776c0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:36.110995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07bcd7-8f7f-4071-9b7b-150b15f776c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:36.112130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07bcd7-8f7f-4071-9b7b-150b15f776c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:36.113375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07bcd7-8f7f-4071-9b7b-150b15f776c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:36.113876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07bcd7-8f7f-4071-9b7b-150b15f776c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:36.115662 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07bcd7-8f7f-4071-9b7b-150b15f776c0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:36.126037 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 85/172] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:36.132571 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a015a2-bcd3-479b-b1ff-83b901cb7eaa None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:36.133200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a015a2-bcd3-479b-b1ff-83b901cb7eaa None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:36.133764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a015a2-bcd3-479b-b1ff-83b901cb7eaa None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:36.135779 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 86/173] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:36.137102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a015a2-bcd3-479b-b1ff-83b901cb7eaa None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:36.176242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 88/174] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:41:36.177898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef556f0-d40c-407e-835f-bbae6db63f19 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:36.178667 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef556f0-d40c-407e-835f-bbae6db63f19 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:36.178957 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef556f0-d40c-407e-835f-bbae6db63f19 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:36.184161 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 89/175] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:36.186009 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef556f0-d40c-407e-835f-bbae6db63f19 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:36.248429 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:36.269730 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a015a2-bcd3-479b-b1ff-83b901cb7eaa None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:36.307658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202db9af-fb33-43e4-a13d-90c29b3ace7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:36.308292 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202db9af-fb33-43e4-a13d-90c29b3ace7f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:36.308579 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202db9af-fb33-43e4-a13d-90c29b3ace7f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:36.308871 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202db9af-fb33-43e4-a13d-90c29b3ace7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:36.309585 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202db9af-fb33-43e4-a13d-90c29b3ace7f None admin] RBAC: Adding query filter params (role.id=6ebbed85905048a198e4261a3510dafc, scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5, user.id=3d9981f1b2c1421b8d5a89485cc87def) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:36.309883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202db9af-fb33-43e4-a13d-90c29b3ace7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:36.311341 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202db9af-fb33-43e4-a13d-90c29b3ace7f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:36.317239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef556f0-d40c-407e-835f-bbae6db63f19 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:36.317730 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 87/176] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:36 2022] GET /identity/v3/role_assignments?user.id=3d9981f1b2c1421b8d5a89485cc87def&scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5&role.id=6ebbed85905048a198e4261a3510dafc => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:36.318951 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be4843-9982-4371-99ed-8c1de9e8e6de None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:36.319233 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be4843-9982-4371-99ed-8c1de9e8e6de None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:36.319505 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be4843-9982-4371-99ed-8c1de9e8e6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:36.339229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-10be4843-9982-4371-99ed-8c1de9e8e6de None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:36.340648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-10be4843-9982-4371-99ed-8c1de9e8e6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:36.365173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d2b33-7553-4612-bdb0-0e5cecc730d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:36.366301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d2b33-7553-4612-bdb0-0e5cecc730d4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:36.366583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d2b33-7553-4612-bdb0-0e5cecc730d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:36.366865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d2b33-7553-4612-bdb0-0e5cecc730d4 None admin] PATH_INFO: `/v3/users/3d9981f1b2c1421b8d5a89485cc87def` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:36.368115 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d2b33-7553-4612-bdb0-0e5cecc730d4 None admin] RBAC: Authorizing `identity:get_user(user_id=3d9981f1b2c1421b8d5a89485cc87def)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:36.369620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d2b33-7553-4612-bdb0-0e5cecc730d4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:36.375972 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 90/177] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:36 2022] GET /identity/v3/users/3d9981f1b2c1421b8d5a89485cc87def => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:36.375972 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d2b33-7553-4612-bdb0-0e5cecc730d4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:36.400578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:36.449001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 88/178] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:36.450295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040c0c6-d796-4157-baf9-58f66fc41a74 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:36.450658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040c0c6-d796-4157-baf9-58f66fc41a74 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:36.450848 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040c0c6-d796-4157-baf9-58f66fc41a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:36.474015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f040c0c6-d796-4157-baf9-58f66fc41a74 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:36.475212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f040c0c6-d796-4157-baf9-58f66fc41a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:36.532755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d2b33-7553-4612-bdb0-0e5cecc730d4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:36.554882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 89/179] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:36.556416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc4b3b-99da-4604-a124-76862335547d None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:36.556767 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc4b3b-99da-4604-a124-76862335547d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:36.557222 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc4b3b-99da-4604-a124-76862335547d None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:36.559034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 90/180] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:36.559809 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc4b3b-99da-4604-a124-76862335547d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:36.576821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:36.588517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd23a9-d6e6-4c2e-b35b-dc257134add9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:36.589881 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd23a9-d6e6-4c2e-b35b-dc257134add9 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:36.590365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd23a9-d6e6-4c2e-b35b-dc257134add9 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:36.590829 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd23a9-d6e6-4c2e-b35b-dc257134add9 None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:36.592234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd23a9-d6e6-4c2e-b35b-dc257134add9 None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:36.594759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd23a9-d6e6-4c2e-b35b-dc257134add9 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:36.598110 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 91/181] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:36 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:36.599721 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b055e-44ae-40ef-9d6c-837b9dc1ccda None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:36.600582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b055e-44ae-40ef-9d6c-837b9dc1ccda None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:36.601068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b055e-44ae-40ef-9d6c-837b9dc1ccda None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:36.602484 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 92/182] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:36.603705 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a848fa6-5d71-48b4-bd8b-1717b5f6af1c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:36.604297 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a848fa6-5d71-48b4-bd8b-1717b5f6af1c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:36.604789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a848fa6-5d71-48b4-bd8b-1717b5f6af1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:36.627569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5a848fa6-5d71-48b4-bd8b-1717b5f6af1c None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:36.629532 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5a848fa6-5d71-48b4-bd8b-1717b5f6af1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:36.704156 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc4b3b-99da-4604-a124-76862335547d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:36.711344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 93/183] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:36.715858 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563f2ae-afcf-4b27-975d-cd0dc4906d5b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:36.715858 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563f2ae-afcf-4b27-975d-cd0dc4906d5b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:36.715858 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563f2ae-afcf-4b27-975d-cd0dc4906d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:36.745110 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a563f2ae-afcf-4b27-975d-cd0dc4906d5b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:36.746307 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a563f2ae-afcf-4b27-975d-cd0dc4906d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:36.753010 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:36.776755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c81dc7-b8f7-4469-ae5f-c5c7003f18cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:36.777602 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c81dc7-b8f7-4469-ae5f-c5c7003f18cc None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:36.777877 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c81dc7-b8f7-4469-ae5f-c5c7003f18cc None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:36.779016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c81dc7-b8f7-4469-ae5f-c5c7003f18cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:36.851480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 94/184] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:36.852351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563f2ae-afcf-4b27-975d-cd0dc4906d5b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:36.877139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c81dc7-b8f7-4469-ae5f-c5c7003f18cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:36.878627 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c81dc7-b8f7-4469-ae5f-c5c7003f18cc None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:36.893205 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:36.991025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563f2ae-afcf-4b27-975d-cd0dc4906d5b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:36.997669 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 91/185] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:41:36.998596 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c81dc7-b8f7-4469-ae5f-c5c7003f18cc None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:37.031244 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f096e-22c1-4462-bb40-a3bf70cdf51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:37.032229 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f096e-22c1-4462-bb40-a3bf70cdf51c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:37.032708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f096e-22c1-4462-bb40-a3bf70cdf51c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:37.033721 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f096e-22c1-4462-bb40-a3bf70cdf51c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:37.034648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f096e-22c1-4462-bb40-a3bf70cdf51c None admin] RBAC: Adding query filter params (role.id=84a30faea3e94eb48b03ec7ceb7c1392, scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5, user.id=3d9981f1b2c1421b8d5a89485cc87def) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:37.035163 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f096e-22c1-4462-bb40-a3bf70cdf51c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:37.036819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f096e-22c1-4462-bb40-a3bf70cdf51c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:37.045451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 95/186] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:36 2022] GET /identity/v3/role_assignments?user.id=3d9981f1b2c1421b8d5a89485cc87def&scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5&role.id=84a30faea3e94eb48b03ec7ceb7c1392 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:37.052107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f096e-22c1-4462-bb40-a3bf70cdf51c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:37.059654 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:37.141988 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c81dc7-b8f7-4469-ae5f-c5c7003f18cc None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:37.153541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:37.182095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a94ca-2a6c-4e31-8b36-829c5b71d1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:37.182853 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a94ca-2a6c-4e31-8b36-829c5b71d1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:37.182987 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f096e-22c1-4462-bb40-a3bf70cdf51c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:37.183413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a94ca-2a6c-4e31-8b36-829c5b71d1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:37.183413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a94ca-2a6c-4e31-8b36-829c5b71d1c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:37.196632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a94ca-2a6c-4e31-8b36-829c5b71d1c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:37.201639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-9b8a94ca-2a6c-4e31-8b36-829c5b71d1c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:41:37.201639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:37.201639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:37.201639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:37.201639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:37.201639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:37.201639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:37.201639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:37.201639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:37.201639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:37.201639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:37.201639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:37.201639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:37.201639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:37.201639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:37.201639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:37.202555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:37.202555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:37.202555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:37.202555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:37.202555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:37.202555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:37.202555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:37.202555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:37.202555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:37.202555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:37.202555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:37.202555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:37.202555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:37.202555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:37.202555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:37.202555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:37.203339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:37.203339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:37.203339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:37.203339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:37.203339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:37.203339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:37.203339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:37.203339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:37.203339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:37.203339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:37.203339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:41:37.203339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:37.203339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 92/187] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:37.204019 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a94ca-2a6c-4e31-8b36-829c5b71d1c9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:37.229064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f2ca7-2b12-4594-a952-8212fa9c9c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:37.229697 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f2ca7-2b12-4594-a952-8212fa9c9c31 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:37.229971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f2ca7-2b12-4594-a952-8212fa9c9c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:37.230242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f2ca7-2b12-4594-a952-8212fa9c9c31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:37.243603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f2ca7-2b12-4594-a952-8212fa9c9c31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:37.249723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-327f2ca7-2b12-4594-a952-8212fa9c9c31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:37.249723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:37.249723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:37.249723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:37.249723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:37.249723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:37.249723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:37.249723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:37.249723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:37.249723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:37.249723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:37.249723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:37.249723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:37.249723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:37.249723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:37.249723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:37.250659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:37.250659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:37.250659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:37.250659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:37.250659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:37.250659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:37.250659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:37.250659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:37.250659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:37.250659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:37.250659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:37.250659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:37.250659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:37.250659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:37.250659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:37.250659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:37.251447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:37.251447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:37.251447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:37.251447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:37.251447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:37.251447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:37.251447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:37.251447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:37.251447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:37.251447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:37.251447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:37.251447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:37.251447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 96/188] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:41:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:37.252314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f2ca7-2b12-4594-a952-8212fa9c9c31 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:37.325737 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:37.337116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a94ca-2a6c-4e31-8b36-829c5b71d1c9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:37.363465 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f2ca7-2b12-4594-a952-8212fa9c9c31 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:37.371282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d9d4b-84c7-439d-8b60-4d4a7c99df1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:37.371905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d9d4b-84c7-439d-8b60-4d4a7c99df1b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:37.372212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d9d4b-84c7-439d-8b60-4d4a7c99df1b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:37.372487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d9d4b-84c7-439d-8b60-4d4a7c99df1b None admin] PATH_INFO: `/v3/projects/bfab41741dcd4147ae89996789a3dbaa` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:37.373608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d9d4b-84c7-439d-8b60-4d4a7c99df1b None admin] RBAC: Authorizing `identity:get_project(project_id=bfab41741dcd4147ae89996789a3dbaa)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:37.375048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d9d4b-84c7-439d-8b60-4d4a7c99df1b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:37.377532 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 93/189] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:37 2022] GET /identity/v3/projects/bfab41741dcd4147ae89996789a3dbaa => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:37.378310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d9d4b-84c7-439d-8b60-4d4a7c99df1b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:37.405057 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589d9e1-8fd3-4059-8bf7-8b004264cdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:37.405627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589d9e1-8fd3-4059-8bf7-8b004264cdbc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:37.405906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589d9e1-8fd3-4059-8bf7-8b004264cdbc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:37.406174 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589d9e1-8fd3-4059-8bf7-8b004264cdbc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:37.407262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589d9e1-8fd3-4059-8bf7-8b004264cdbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:37.408763 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589d9e1-8fd3-4059-8bf7-8b004264cdbc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:37.410309 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 97/190] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:37.413325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589d9e1-8fd3-4059-8bf7-8b004264cdbc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:37.413455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:37.481948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d9d4b-84c7-439d-8b60-4d4a7c99df1b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:37.512551 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589d9e1-8fd3-4059-8bf7-8b004264cdbc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:37.516069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23c720-48c3-4ac6-b8b2-f79c6c22c9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:37.516811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23c720-48c3-4ac6-b8b2-f79c6c22c9ae None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:37.516998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23c720-48c3-4ac6-b8b2-f79c6c22c9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:37.517272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23c720-48c3-4ac6-b8b2-f79c6c22c9ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:37.527645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23c720-48c3-4ac6-b8b2-f79c6c22c9ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:37.532506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-5d23c720-48c3-4ac6-b8b2-f79c6c22c9ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:41:37.532506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:37.532506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:37.532506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:37.532506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:37.532506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:37.532506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:37.532506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:37.532506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:37.532506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:37.532506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:37.532506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:37.532506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:37.532506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:37.532506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:37.532506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:37.533488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:37.533488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:37.533488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:37.533488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:37.533488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:37.533488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:37.533488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:37.533488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:37.533488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:37.533488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:37.533488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:37.533488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:37.533488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:37.533488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:37.533488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:37.533488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:37.534337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:37.534337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:37.534337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:37.534337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:37.534337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:37.534337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:37.534337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:37.534337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:37.534337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:37.534337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:37.534337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:41:37.534337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:37.534337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 94/191] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:41:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:37.536903 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23c720-48c3-4ac6-b8b2-f79c6c22c9ae None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:37.545811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:37.551657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac2f53-716a-4ddd-a115-ca7716eb53ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:37.552291 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac2f53-716a-4ddd-a115-ca7716eb53ce None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:37.552566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac2f53-716a-4ddd-a115-ca7716eb53ce None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:37.552834 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac2f53-716a-4ddd-a115-ca7716eb53ce None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:37.553486 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac2f53-716a-4ddd-a115-ca7716eb53ce None admin] RBAC: Adding query filter params (name=reader) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:37.553783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac2f53-716a-4ddd-a115-ca7716eb53ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:37.555131 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac2f53-716a-4ddd-a115-ca7716eb53ce None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:37.563184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 98/192] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:37.564135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac2f53-716a-4ddd-a115-ca7716eb53ce None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:37.638223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:37.645731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23c720-48c3-4ac6-b8b2-f79c6c22c9ae None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:37.665044 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac2f53-716a-4ddd-a115-ca7716eb53ce None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:37.681023 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6faf6c-1efd-4d99-948b-e7d7648866d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:37.681621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6faf6c-1efd-4d99-948b-e7d7648866d7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:37.681890 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6faf6c-1efd-4d99-948b-e7d7648866d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:37.682157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6faf6c-1efd-4d99-948b-e7d7648866d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:37.682810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6faf6c-1efd-4d99-948b-e7d7648866d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:37.683106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6faf6c-1efd-4d99-948b-e7d7648866d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:37.684527 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6faf6c-1efd-4d99-948b-e7d7648866d7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:37.693331 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 95/193] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:37.694349 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6faf6c-1efd-4d99-948b-e7d7648866d7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:37.699080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af54cd-7d57-446a-87e3-8540d3fe1c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:37.699683 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af54cd-7d57-446a-87e3-8540d3fe1c55 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:37.699975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af54cd-7d57-446a-87e3-8540d3fe1c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:37.700270 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af54cd-7d57-446a-87e3-8540d3fe1c55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:37.700993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af54cd-7d57-446a-87e3-8540d3fe1c55 None admin] RBAC: Adding query filter params (role.id=6ebbed85905048a198e4261a3510dafc, scope.project.id=bfab41741dcd4147ae89996789a3dbaa, user.id=3d9981f1b2c1421b8d5a89485cc87def) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:37.701298 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af54cd-7d57-446a-87e3-8540d3fe1c55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:37.702714 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af54cd-7d57-446a-87e3-8540d3fe1c55 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:37.708627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 99/194] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:37 2022] GET /identity/v3/role_assignments?user.id=3d9981f1b2c1421b8d5a89485cc87def&scope.project.id=bfab41741dcd4147ae89996789a3dbaa&role.id=6ebbed85905048a198e4261a3510dafc => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:37.718629 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af54cd-7d57-446a-87e3-8540d3fe1c55 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:37.786322 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:37.806014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6faf6c-1efd-4d99-948b-e7d7648866d7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:37.835122 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af54cd-7d57-446a-87e3-8540d3fe1c55 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:37.846291 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096a1c6-5060-491e-9c86-ccd0bd19848c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:37.847399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096a1c6-5060-491e-9c86-ccd0bd19848c None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:37.847684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096a1c6-5060-491e-9c86-ccd0bd19848c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:37.848106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096a1c6-5060-491e-9c86-ccd0bd19848c None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:37.848944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096a1c6-5060-491e-9c86-ccd0bd19848c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:37.850655 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096a1c6-5060-491e-9c86-ccd0bd19848c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:37.882058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 96/195] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:41:37 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:41:37.882058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096a1c6-5060-491e-9c86-ccd0bd19848c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:37.883065 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29074072-c92c-44bc-af3d-7e440d075da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:37.883838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29074072-c92c-44bc-af3d-7e440d075da6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:37.884216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29074072-c92c-44bc-af3d-7e440d075da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:37.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29074072-c92c-44bc-af3d-7e440d075da6 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:37.885193 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29074072-c92c-44bc-af3d-7e440d075da6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:37.885494 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29074072-c92c-44bc-af3d-7e440d075da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:37.886932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29074072-c92c-44bc-af3d-7e440d075da6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:37.898852 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 100/196] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:41:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:37.899855 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29074072-c92c-44bc-af3d-7e440d075da6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:37.904843 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:38.037434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096a1c6-5060-491e-9c86-ccd0bd19848c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:38.046380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29074072-c92c-44bc-af3d-7e440d075da6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:38.070419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:38.080710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592c73a-2d44-4059-92f1-c69be21e98aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:38.081628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592c73a-2d44-4059-92f1-c69be21e98aa None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:38.081978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592c73a-2d44-4059-92f1-c69be21e98aa None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:38.082326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592c73a-2d44-4059-92f1-c69be21e98aa None admin] PATH_INFO: `/v3/users/a9894cf5ac414bcfa637feb6d3fd1943` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:38.083665 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592c73a-2d44-4059-92f1-c69be21e98aa None admin] RBAC: Authorizing `identity:get_user(user_id=a9894cf5ac414bcfa637feb6d3fd1943)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:38.085179 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592c73a-2d44-4059-92f1-c69be21e98aa None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:38.087132 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 97/197] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:37 2022] GET /identity/v3/users/a9894cf5ac414bcfa637feb6d3fd1943 => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:38.087747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592c73a-2d44-4059-92f1-c69be21e98aa None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:38.094561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0182226-3666-42a4-8a7d-982074c99085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:38.095204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0182226-3666-42a4-8a7d-982074c99085 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:38.095485 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0182226-3666-42a4-8a7d-982074c99085 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:38.096330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0182226-3666-42a4-8a7d-982074c99085 None admin] PATH_INFO: `/v3/users/db5dc2d21805484b9ef54fb44f746191` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:38.117476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0182226-3666-42a4-8a7d-982074c99085 None admin] RBAC: Authorizing `identity:get_user(user_id=db5dc2d21805484b9ef54fb44f746191)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:38.119009 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0182226-3666-42a4-8a7d-982074c99085 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:38.125256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 101/198] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:37 2022] GET /identity/v3/users/db5dc2d21805484b9ef54fb44f746191 => generated 283 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:38.126110 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0182226-3666-42a4-8a7d-982074c99085 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:38.187070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:38.207820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592c73a-2d44-4059-92f1-c69be21e98aa None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:38.253273 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d7074-650c-4038-a752-36f4c95692b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:38.254180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d7074-650c-4038-a752-36f4c95692b9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:38.254551 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d7074-650c-4038-a752-36f4c95692b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:38.254913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d7074-650c-4038-a752-36f4c95692b9 None admin] PATH_INFO: `/v3/users/db5dc2d21805484b9ef54fb44f746191` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:38.256196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d7074-650c-4038-a752-36f4c95692b9 None admin] RBAC: Authorizing `identity:get_user(user_id=db5dc2d21805484b9ef54fb44f746191)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:38.257274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0182226-3666-42a4-8a7d-982074c99085 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:38.257800 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d7074-650c-4038-a752-36f4c95692b9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:38.259505 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 98/199] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:38 2022] GET /identity/v3/users/db5dc2d21805484b9ef54fb44f746191 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:38.260206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d7074-650c-4038-a752-36f4c95692b9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:38.317292 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade034f-369d-4e10-80d1-74d98c08f81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:38.318101 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade034f-369d-4e10-80d1-74d98c08f81c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:38.318386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade034f-369d-4e10-80d1-74d98c08f81c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:38.318913 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade034f-369d-4e10-80d1-74d98c08f81c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:38.368974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:38.381874 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d7074-650c-4038-a752-36f4c95692b9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:38.395723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade034f-369d-4e10-80d1-74d98c08f81c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:38.397242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade034f-369d-4e10-80d1-74d98c08f81c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:38.423147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c28a2d-9fbc-4879-aea9-683504d74795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:38.423797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c28a2d-9fbc-4879-aea9-683504d74795 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:38.424070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c28a2d-9fbc-4879-aea9-683504d74795 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:38.424378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c28a2d-9fbc-4879-aea9-683504d74795 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:38.445290 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:38.501796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c28a2d-9fbc-4879-aea9-683504d74795 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:38.503565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c28a2d-9fbc-4879-aea9-683504d74795 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:38.504638 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 102/200] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:41:38.505489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade034f-369d-4e10-80d1-74d98c08f81c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:38.594833 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:38.616113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 99/201] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:41:38.617997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c28a2d-9fbc-4879-aea9-683504d74795 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:38.623346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade034f-369d-4e10-80d1-74d98c08f81c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:38.659339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6d9f7-d6a6-4b58-bda5-ca511ab9bd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:38.660060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6d9f7-d6a6-4b58-bda5-ca511ab9bd4c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:38.660406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6d9f7-d6a6-4b58-bda5-ca511ab9bd4c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:38.660629 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6d9f7-d6a6-4b58-bda5-ca511ab9bd4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:38.680758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:38.736933 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c28a2d-9fbc-4879-aea9-683504d74795 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:38.740351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6d9f7-d6a6-4b58-bda5-ca511ab9bd4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:38.741581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6d9f7-d6a6-4b58-bda5-ca511ab9bd4c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:38.774598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3041bc-8a4b-4e20-969f-7407e582c02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:38.775479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3041bc-8a4b-4e20-969f-7407e582c02c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:38.775917 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3041bc-8a4b-4e20-969f-7407e582c02c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:38.776353 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3041bc-8a4b-4e20-969f-7407e582c02c None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:38.777610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3041bc-8a4b-4e20-969f-7407e582c02c None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:38.779271 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3041bc-8a4b-4e20-969f-7407e582c02c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:38.781092 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 100/202] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:38 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:38.782061 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3041bc-8a4b-4e20-969f-7407e582c02c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:38.824713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:38.850386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 103/203] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:41:38.851013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6d9f7-d6a6-4b58-bda5-ca511ab9bd4c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:38.890383 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3041bc-8a4b-4e20-969f-7407e582c02c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:38.909102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:38.932066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211aa179-323f-444d-90b6-964d04187477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:38.932994 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211aa179-323f-444d-90b6-964d04187477 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:38.933412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211aa179-323f-444d-90b6-964d04187477 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:38.933822 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211aa179-323f-444d-90b6-964d04187477 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:38.935108 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211aa179-323f-444d-90b6-964d04187477 None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:38.936740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211aa179-323f-444d-90b6-964d04187477 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:38.938551 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 101/204] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:38 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:38.940225 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211aa179-323f-444d-90b6-964d04187477 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:38.965404 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6d9f7-d6a6-4b58-bda5-ca511ab9bd4c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:39.002084 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a26dc-52af-4e69-bd1d-340ecace7b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:39.002769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a26dc-52af-4e69-bd1d-340ecace7b61 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:39.003113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a26dc-52af-4e69-bd1d-340ecace7b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:39.003389 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a26dc-52af-4e69-bd1d-340ecace7b61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:39.004190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a26dc-52af-4e69-bd1d-340ecace7b61 None admin] RBAC: Adding query filter params (role.id=a5350d487c374ea680a13ac3044168fc, scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5, user.id=a9894cf5ac414bcfa637feb6d3fd1943) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:39.004453 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a26dc-52af-4e69-bd1d-340ecace7b61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:39.005896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a26dc-52af-4e69-bd1d-340ecace7b61 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:39.012066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 104/205] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:38 2022] GET /identity/v3/role_assignments?user.id=a9894cf5ac414bcfa637feb6d3fd1943&scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5&role.id=a5350d487c374ea680a13ac3044168fc => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:39.012999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a26dc-52af-4e69-bd1d-340ecace7b61 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:39.058221 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211aa179-323f-444d-90b6-964d04187477 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:39.065285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:39.098237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bf5d4-0510-4a64-b01c-6eb04be4b284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:39.099150 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bf5d4-0510-4a64-b01c-6eb04be4b284 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:39.099567 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bf5d4-0510-4a64-b01c-6eb04be4b284 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:39.099976 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bf5d4-0510-4a64-b01c-6eb04be4b284 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:39.101319 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8bf5d4-0510-4a64-b01c-6eb04be4b284 None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:39.102996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8bf5d4-0510-4a64-b01c-6eb04be4b284 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:39.104860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 102/206] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:38 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:39.116101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bf5d4-0510-4a64-b01c-6eb04be4b284 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:39.157507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a26dc-52af-4e69-bd1d-340ecace7b61 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:39.229569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2e32a-63f8-4a96-adc9-6a37eb9a0699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:39.230356 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2e32a-63f8-4a96-adc9-6a37eb9a0699 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:39.230732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2e32a-63f8-4a96-adc9-6a37eb9a0699 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:39.231067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2e32a-63f8-4a96-adc9-6a37eb9a0699 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:39.231810 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2e32a-63f8-4a96-adc9-6a37eb9a0699 None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=ea579b172f214888a95d9af5a4e1d122, user.id=db5dc2d21805484b9ef54fb44f746191) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:39.232191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2e32a-63f8-4a96-adc9-6a37eb9a0699 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:39.233769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2e32a-63f8-4a96-adc9-6a37eb9a0699 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:39.246717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:39.250971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 105/207] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:39 2022] GET /identity/v3/role_assignments?user.id=db5dc2d21805484b9ef54fb44f746191&scope.project.id=ea579b172f214888a95d9af5a4e1d122&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:39.308111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bf5d4-0510-4a64-b01c-6eb04be4b284 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:39.359929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd2e07-cf93-4b61-85f3-fe10b88f0846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:39.359929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd2e07-cf93-4b61-85f3-fe10b88f0846 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:39.359929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd2e07-cf93-4b61-85f3-fe10b88f0846 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:39.359929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd2e07-cf93-4b61-85f3-fe10b88f0846 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:39.360327 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd2e07-cf93-4b61-85f3-fe10b88f0846 None admin] RBAC: Adding query filter params (role.id=84a30faea3e94eb48b03ec7ceb7c1392, scope.project.id=ea579b172f214888a95d9af5a4e1d122, user.id=db5dc2d21805484b9ef54fb44f746191) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:39.360327 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd2e07-cf93-4b61-85f3-fe10b88f0846 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:39.360327 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd2e07-cf93-4b61-85f3-fe10b88f0846 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:39.369399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 103/208] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:39 2022] GET /identity/v3/role_assignments?user.id=db5dc2d21805484b9ef54fb44f746191&scope.project.id=ea579b172f214888a95d9af5a4e1d122&role.id=84a30faea3e94eb48b03ec7ceb7c1392 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:40.420627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe63a8-f71f-4da0-82a8-ed234380b95b None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:40.421157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe63a8-f71f-4da0-82a8-ed234380b95b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:40.421157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe63a8-f71f-4da0-82a8-ed234380b95b None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:40.422679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 106/209] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:40.424955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b42ca-aa7d-43b7-ab07-10dd2080fde2 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:40.425563 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b42ca-aa7d-43b7-ab07-10dd2080fde2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:40.425751 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b42ca-aa7d-43b7-ab07-10dd2080fde2 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:40.427546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 104/210] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:40.433319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0229de1-c227-4777-9793-2c6a9328d67a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:40.433618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0229de1-c227-4777-9793-2c6a9328d67a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:40.433875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0229de1-c227-4777-9793-2c6a9328d67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:40.441133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656dba37-56ff-4af6-ba8c-31194687004f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:40.441133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656dba37-56ff-4af6-ba8c-31194687004f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:40.441133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656dba37-56ff-4af6-ba8c-31194687004f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:40.453737 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c0229de1-c227-4777-9793-2c6a9328d67a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:40.455158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c0229de1-c227-4777-9793-2c6a9328d67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:40.457529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-656dba37-56ff-4af6-ba8c-31194687004f None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:40.458850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-656dba37-56ff-4af6-ba8c-31194687004f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:40.532117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 107/211] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:40.541839 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 105/212] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:40.802093 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0229de1-c227-4777-9793-2c6a9328d67a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:40.816282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656dba37-56ff-4af6-ba8c-31194687004f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:40.842841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:40.887335 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:40.917802 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0229de1-c227-4777-9793-2c6a9328d67a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:40.959071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656dba37-56ff-4af6-ba8c-31194687004f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:40.959251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4190cc9-9358-4207-9795-24ef99fd8c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:40.959857 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4190cc9-9358-4207-9795-24ef99fd8c21 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:40.960172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4190cc9-9358-4207-9795-24ef99fd8c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:40.960450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4190cc9-9358-4207-9795-24ef99fd8c21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:40.971877 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4190cc9-9358-4207-9795-24ef99fd8c21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:40.976710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-c4190cc9-9358-4207-9795-24ef99fd8c21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:40.976710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:40.976710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:40.976710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:40.976710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:40.976710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:40.976710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:40.976710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:40.976710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:40.976710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:40.976710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:40.976710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:40.976710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:40.976710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:40.976710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:40.976710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:40.977569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:40.977569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:40.977569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:40.977569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:40.977569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:40.977569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:40.977569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:40.977569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:40.977569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:40.977569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:40.977569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:40.977569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:40.977569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:40.977569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:40.977569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:40.977569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:40.978342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:40.978342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:40.978342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:40.978342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:40.978342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:40.978342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:40.978342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:40.978342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:40.978342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:40.978342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:40.978342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:40.978342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:40.978342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 108/213] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:40.985056 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4190cc9-9358-4207-9795-24ef99fd8c21 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:41.011273 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaaa0bc-fc94-410b-aad7-cf7bfadcb67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:41.012558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaa0bc-fc94-410b-aad7-cf7bfadcb67a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:41.012983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaa0bc-fc94-410b-aad7-cf7bfadcb67a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:41.013654 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaa0bc-fc94-410b-aad7-cf7bfadcb67a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:41.015152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaaa0bc-fc94-410b-aad7-cf7bfadcb67a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:41.017433 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaaa0bc-fc94-410b-aad7-cf7bfadcb67a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:41.020631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 106/214] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:41.026036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaaa0bc-fc94-410b-aad7-cf7bfadcb67a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:41.091828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:41.100286 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4190cc9-9358-4207-9795-24ef99fd8c21 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:41.132666 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaaa0bc-fc94-410b-aad7-cf7bfadcb67a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:41.137708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1448db6-9379-4732-8a72-4a8c3968a9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:41.138321 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1448db6-9379-4732-8a72-4a8c3968a9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:41.138597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1448db6-9379-4732-8a72-4a8c3968a9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:41.138870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1448db6-9379-4732-8a72-4a8c3968a9e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:41.139537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1448db6-9379-4732-8a72-4a8c3968a9e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:41.139838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1448db6-9379-4732-8a72-4a8c3968a9e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:41.141285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1448db6-9379-4732-8a72-4a8c3968a9e4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:41.156399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:41.156569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 109/215] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:41.159052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1448db6-9379-4732-8a72-4a8c3968a9e4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:41.172724 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e3153-4eba-4eef-a448-35868abc2f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:41.173632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e3153-4eba-4eef-a448-35868abc2f64 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:41.174327 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e3153-4eba-4eef-a448-35868abc2f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:41.174984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e3153-4eba-4eef-a448-35868abc2f64 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:41.175830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e3153-4eba-4eef-a448-35868abc2f64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:41.177503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e3153-4eba-4eef-a448-35868abc2f64 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:41.216125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 107/216] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:41:41 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:41:41.284497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1448db6-9379-4732-8a72-4a8c3968a9e4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:41.328571 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db4112-8f7e-455b-9722-dc9dbd03f0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:41.329702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db4112-8f7e-455b-9722-dc9dbd03f0be None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:41.330732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db4112-8f7e-455b-9722-dc9dbd03f0be None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:41.331212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db4112-8f7e-455b-9722-dc9dbd03f0be None admin] PATH_INFO: `/v3/users/b83b7b6bbea849d4b526b03c27564c2f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:41.347913 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db4112-8f7e-455b-9722-dc9dbd03f0be None admin] RBAC: Authorizing `identity:get_user(user_id=b83b7b6bbea849d4b526b03c27564c2f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:41.349861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db4112-8f7e-455b-9722-dc9dbd03f0be None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:41.351889 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 110/217] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:41 2022] GET /identity/v3/users/b83b7b6bbea849d4b526b03c27564c2f => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:41.360182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e3153-4eba-4eef-a448-35868abc2f64 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:41.472993 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e3153-4eba-4eef-a448-35868abc2f64 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:41.511022 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948e0c1-22f7-4860-8291-ccbca81b3f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:41.511638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948e0c1-22f7-4860-8291-ccbca81b3f00 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:41.511920 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948e0c1-22f7-4860-8291-ccbca81b3f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:41.512227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948e0c1-22f7-4860-8291-ccbca81b3f00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:41.563527 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:41.599802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948e0c1-22f7-4860-8291-ccbca81b3f00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:41.601277 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948e0c1-22f7-4860-8291-ccbca81b3f00 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:41.725420 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 108/218] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:41:41.731021 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db4112-8f7e-455b-9722-dc9dbd03f0be None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:41.738134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3c470-cadd-4a26-8ce4-54082d8aa19e None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:41.738410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3c470-cadd-4a26-8ce4-54082d8aa19e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:41.738688 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3c470-cadd-4a26-8ce4-54082d8aa19e None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:41.740325 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 109/219] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:41.746857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98429c7e-6d59-48e6-9369-99c002348264 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:41.747137 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98429c7e-6d59-48e6-9369-99c002348264 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:41.747985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98429c7e-6d59-48e6-9369-99c002348264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:41.761600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:41.773015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-98429c7e-6d59-48e6-9369-99c002348264 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:41.774372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-98429c7e-6d59-48e6-9369-99c002348264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:41.860189 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db4112-8f7e-455b-9722-dc9dbd03f0be None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:41.860358 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 110/220] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:41.868103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2443e46-a5db-44c3-b07c-f39db681bd75 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:41.868103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2443e46-a5db-44c3-b07c-f39db681bd75 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:41.868103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2443e46-a5db-44c3-b07c-f39db681bd75 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:41.868103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 111/221] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:41.869914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea689b2-3360-4c4e-ad00-f3c24aa9cd04 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:41.870259 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea689b2-3360-4c4e-ad00-f3c24aa9cd04 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:41.870600 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea689b2-3360-4c4e-ad00-f3c24aa9cd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:41.878195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:41.891427 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3ea689b2-3360-4c4e-ad00-f3c24aa9cd04 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:41.893477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3ea689b2-3360-4c4e-ad00-f3c24aa9cd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:41.968133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 112/222] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:41.976806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14247cd4-f074-4e1e-931b-2a7537cc9ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:41.977676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14247cd4-f074-4e1e-931b-2a7537cc9ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:41.978124 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14247cd4-f074-4e1e-931b-2a7537cc9ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:41.978564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14247cd4-f074-4e1e-931b-2a7537cc9ee2 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:41.984752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14247cd4-f074-4e1e-931b-2a7537cc9ee2 None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:41.988179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14247cd4-f074-4e1e-931b-2a7537cc9ee2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:41.990019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 111/223] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:41 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:41.996142 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea689b2-3360-4c4e-ad00-f3c24aa9cd04 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:42.154855 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea689b2-3360-4c4e-ad00-f3c24aa9cd04 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:42.185988 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14247cd4-f074-4e1e-931b-2a7537cc9ee2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:42.198678 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:42.207067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32155e09-9363-41c4-87f1-90b11f4d2407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:42.207949 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32155e09-9363-41c4-87f1-90b11f4d2407 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:42.208439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32155e09-9363-41c4-87f1-90b11f4d2407 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:42.208977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32155e09-9363-41c4-87f1-90b11f4d2407 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:42.210625 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32155e09-9363-41c4-87f1-90b11f4d2407 None admin] RBAC: Adding query filter params (role.id=6ebbed85905048a198e4261a3510dafc, scope.project.id=ea579b172f214888a95d9af5a4e1d122, user.id=b83b7b6bbea849d4b526b03c27564c2f) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:42.211288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32155e09-9363-41c4-87f1-90b11f4d2407 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:42.214249 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32155e09-9363-41c4-87f1-90b11f4d2407 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:42.226869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 113/224] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:41 2022] GET /identity/v3/role_assignments?user.id=b83b7b6bbea849d4b526b03c27564c2f&scope.project.id=ea579b172f214888a95d9af5a4e1d122&role.id=6ebbed85905048a198e4261a3510dafc => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:42.274492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32155e09-9363-41c4-87f1-90b11f4d2407 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:42.335096 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:42.361879 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14247cd4-f074-4e1e-931b-2a7537cc9ee2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:42.434483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32155e09-9363-41c4-87f1-90b11f4d2407 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:42.440568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b778e70-83a7-453d-ab25-02286c3bc161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:42.441215 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b778e70-83a7-453d-ab25-02286c3bc161 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:42.441489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b778e70-83a7-453d-ab25-02286c3bc161 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:42.447992 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b778e70-83a7-453d-ab25-02286c3bc161 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:42.471676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b778e70-83a7-453d-ab25-02286c3bc161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:42.481959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-2b778e70-83a7-453d-ab25-02286c3bc161 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:42.481959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:42.481959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:42.481959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:42.481959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:42.481959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:42.481959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:42.481959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:42.481959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:42.481959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:42.481959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:42.481959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:42.481959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:42.481959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:42.481959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:42.481959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:42.482905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:42.482905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:42.482905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:42.482905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:42.482905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:42.482905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:42.482905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:42.482905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:42.482905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:42.482905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:42.482905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:42.482905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:42.482905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:42.482905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:42.482905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:42.482905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:42.483669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:42.483669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:42.483669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:42.483669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:42.483669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:42.483669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:42.483669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:42.483669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:42.483669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:42.483669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:42.483669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:42.483669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:42.492780 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 112/225] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:41:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:41:42.493747 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b778e70-83a7-453d-ab25-02286c3bc161 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:42.519657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068b2f5-481b-49e6-94a9-679ccae96c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:42.520365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068b2f5-481b-49e6-94a9-679ccae96c38 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:42.520606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068b2f5-481b-49e6-94a9-679ccae96c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:42.520876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068b2f5-481b-49e6-94a9-679ccae96c38 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:42.532605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068b2f5-481b-49e6-94a9-679ccae96c38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:42.538004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-f068b2f5-481b-49e6-94a9-679ccae96c38 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:41:42.538004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:42.538004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:42.538004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:42.538004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:42.538004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:42.538004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:42.538004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:42.538004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:42.538004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:42.538004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:42.538004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:42.538004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:42.538004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:42.538004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:42.538004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:42.538869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:42.538869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:42.538869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:42.538869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:42.538869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:42.538869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:42.538869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:42.538869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:42.538869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:42.538869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:42.538869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:42.538869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:42.538869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:42.538869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:42.538869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:42.538869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:42.539623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:42.539623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:42.539623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:42.539623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:42.539623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:42.539623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:42.539623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:42.539623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:42.539623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:42.539623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:42.539623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:41:42.539623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:42.539623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 114/226] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:41:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:41:42.551753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068b2f5-481b-49e6-94a9-679ccae96c38 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:42.659326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:42.670966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b778e70-83a7-453d-ab25-02286c3bc161 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:42.713345 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282ade1-0030-47a5-94d9-7dc1f6f7da0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:42.713878 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282ade1-0030-47a5-94d9-7dc1f6f7da0c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:42.714166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282ade1-0030-47a5-94d9-7dc1f6f7da0c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:42.714427 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282ade1-0030-47a5-94d9-7dc1f6f7da0c None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:42.715079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282ade1-0030-47a5-94d9-7dc1f6f7da0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:42.715378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282ade1-0030-47a5-94d9-7dc1f6f7da0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:42.716770 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282ade1-0030-47a5-94d9-7dc1f6f7da0c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:42.733225 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 113/227] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:42.745069 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282ade1-0030-47a5-94d9-7dc1f6f7da0c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:42.746051 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068b2f5-481b-49e6-94a9-679ccae96c38 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:42.767284 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:42.784572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fb5a8-a6a2-4476-b215-f3271e84ff4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:42.785187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fb5a8-a6a2-4476-b215-f3271e84ff4d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:42.786425 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fb5a8-a6a2-4476-b215-f3271e84ff4d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:42.786602 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fb5a8-a6a2-4476-b215-f3271e84ff4d None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:42.787256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fb5a8-a6a2-4476-b215-f3271e84ff4d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:42.787549 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fb5a8-a6a2-4476-b215-f3271e84ff4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:42.788949 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fb5a8-a6a2-4476-b215-f3271e84ff4d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:42.807029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 115/228] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:41:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:42.817036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fb5a8-a6a2-4476-b215-f3271e84ff4d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:42.887334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282ade1-0030-47a5-94d9-7dc1f6f7da0c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:42.929449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ca08b-2923-420d-acfe-480afe45dd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:42.930535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ca08b-2923-420d-acfe-480afe45dd73 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:42.931011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ca08b-2923-420d-acfe-480afe45dd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:42.931480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ca08b-2923-420d-acfe-480afe45dd73 None admin] PATH_INFO: `/v3/users/db5dc2d21805484b9ef54fb44f746191` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:42.938706 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ca08b-2923-420d-acfe-480afe45dd73 None admin] RBAC: Authorizing `identity:get_user(user_id=db5dc2d21805484b9ef54fb44f746191)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:42.940736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ca08b-2923-420d-acfe-480afe45dd73 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:42.942897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 114/229] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:42 2022] GET /identity/v3/users/db5dc2d21805484b9ef54fb44f746191 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:42.949478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ca08b-2923-420d-acfe-480afe45dd73 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:42.968282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fb5a8-a6a2-4476-b215-f3271e84ff4d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:42.980832 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:43.005134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c8162-f6d9-4d5b-9b91-f8349cd0b83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:43.007284 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c8162-f6d9-4d5b-9b91-f8349cd0b83a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:43.008454 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c8162-f6d9-4d5b-9b91-f8349cd0b83a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:43.009515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c8162-f6d9-4d5b-9b91-f8349cd0b83a None admin] PATH_INFO: `/v3/users/db5dc2d21805484b9ef54fb44f746191` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:43.012001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c8162-f6d9-4d5b-9b91-f8349cd0b83a None admin] RBAC: Authorizing `identity:get_user(user_id=db5dc2d21805484b9ef54fb44f746191)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:43.014352 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c8162-f6d9-4d5b-9b91-f8349cd0b83a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:43.016689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 116/230] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:42 2022] GET /identity/v3/users/db5dc2d21805484b9ef54fb44f746191 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:43.025870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c8162-f6d9-4d5b-9b91-f8349cd0b83a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:43.105212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:43.166225 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ca08b-2923-420d-acfe-480afe45dd73 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:43.176534 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c8162-f6d9-4d5b-9b91-f8349cd0b83a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:43.215485 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c656b-ad45-426a-bdc9-79fab9ba72e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:43.216240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c656b-ad45-426a-bdc9-79fab9ba72e2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:43.216591 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c656b-ad45-426a-bdc9-79fab9ba72e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:43.216939 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c656b-ad45-426a-bdc9-79fab9ba72e2 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:43.218409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c656b-ad45-426a-bdc9-79fab9ba72e2 None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:43.220107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c656b-ad45-426a-bdc9-79fab9ba72e2 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:43.222296 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 117/231] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:43 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:43.228731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c656b-ad45-426a-bdc9-79fab9ba72e2 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:43.261345 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbea19-55ca-4043-8120-4cfe96766e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:43.261990 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbea19-55ca-4043-8120-4cfe96766e8f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:43.262262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbea19-55ca-4043-8120-4cfe96766e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:43.262527 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbea19-55ca-4043-8120-4cfe96766e8f None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:43.268883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbea19-55ca-4043-8120-4cfe96766e8f None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:43.270342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbea19-55ca-4043-8120-4cfe96766e8f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:43.272380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 115/232] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:42 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:43.284106 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbea19-55ca-4043-8120-4cfe96766e8f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:43.294217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:43.343658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c656b-ad45-426a-bdc9-79fab9ba72e2 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:43.393712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181658c-4c2c-4598-803e-ac7d9c062167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:43.394397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181658c-4c2c-4598-803e-ac7d9c062167 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:43.394676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181658c-4c2c-4598-803e-ac7d9c062167 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:43.394945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181658c-4c2c-4598-803e-ac7d9c062167 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122/users/db5dc2d21805484b9ef54fb44f746191/roles/84a30faea3e94eb48b03ec7ceb7c1392` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:43.397302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181658c-4c2c-4598-803e-ac7d9c062167 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea579b172f214888a95d9af5a4e1d122, user_id=db5dc2d21805484b9ef54fb44f746191, role_id=84a30faea3e94eb48b03ec7ceb7c1392)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:43.398758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181658c-4c2c-4598-803e-ac7d9c062167 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:43.425539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 118/233] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:41:43 2022] PUT /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122/users/db5dc2d21805484b9ef54fb44f746191/roles/84a30faea3e94eb48b03ec7ceb7c1392 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:41:43.487785 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbea19-55ca-4043-8120-4cfe96766e8f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:43.525090 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e4755-aee0-48eb-afeb-e8a56b0d2766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:43.525764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e4755-aee0-48eb-afeb-e8a56b0d2766 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:43.526042 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e4755-aee0-48eb-afeb-e8a56b0d2766 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:43.526381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e4755-aee0-48eb-afeb-e8a56b0d2766 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122/users/db5dc2d21805484b9ef54fb44f746191/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:43.528375 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e4755-aee0-48eb-afeb-e8a56b0d2766 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea579b172f214888a95d9af5a4e1d122, user_id=db5dc2d21805484b9ef54fb44f746191, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:43.529844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e4755-aee0-48eb-afeb-e8a56b0d2766 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:43.553348 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 116/234] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:41:43 2022] PUT /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122/users/db5dc2d21805484b9ef54fb44f746191/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:41:44.019203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4573b55-72f7-4c50-94fe-e2d80aaee819 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:44.019747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4573b55-72f7-4c50-94fe-e2d80aaee819 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:44.019747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4573b55-72f7-4c50-94fe-e2d80aaee819 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:44.021202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 119/235] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:44.026530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdeb19-4b05-4b07-897a-b483f887ebce None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:44.027175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdeb19-4b05-4b07-897a-b483f887ebce None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:44.027175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdeb19-4b05-4b07-897a-b483f887ebce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:44.033809 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:44.052381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-04fdeb19-4b05-4b07-897a-b483f887ebce None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:44.053594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-04fdeb19-4b05-4b07-897a-b483f887ebce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:44.066073 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a8eb6-78de-4fd6-9ea9-abcd0f2b40fb None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:44.066398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a8eb6-78de-4fd6-9ea9-abcd0f2b40fb None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:44.066622 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a8eb6-78de-4fd6-9ea9-abcd0f2b40fb None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:44.067955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 120/236] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:41:44.074468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c78981-13a8-4624-9f58-4b3acab0efd6 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:44.074745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c78981-13a8-4624-9f58-4b3acab0efd6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:44.075014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c78981-13a8-4624-9f58-4b3acab0efd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:44.101072 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-31c78981-13a8-4624-9f58-4b3acab0efd6 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:44.102512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-31c78981-13a8-4624-9f58-4b3acab0efd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:44.193316 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 117/237] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:44.238849 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 121/238] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:44.544953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdeb19-4b05-4b07-897a-b483f887ebce None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:44.595016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c78981-13a8-4624-9f58-4b3acab0efd6 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:44.650648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:44.691915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdeb19-4b05-4b07-897a-b483f887ebce None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:44.717831 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c78981-13a8-4624-9f58-4b3acab0efd6 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:44.725395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:44.731589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba8ece-5dc5-4f1c-9703-ce5e26748991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:44.732550 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba8ece-5dc5-4f1c-9703-ce5e26748991 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:44.733055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba8ece-5dc5-4f1c-9703-ce5e26748991 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:44.733524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba8ece-5dc5-4f1c-9703-ce5e26748991 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:44.734955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba8ece-5dc5-4f1c-9703-ce5e26748991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:44.737027 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba8ece-5dc5-4f1c-9703-ce5e26748991 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:44.739010 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 118/239] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:44.744661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba8ece-5dc5-4f1c-9703-ce5e26748991 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:44.798159 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49e264-4ba4-4507-bae5-70c49eb344d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:44.799271 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49e264-4ba4-4507-bae5-70c49eb344d3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:44.799869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49e264-4ba4-4507-bae5-70c49eb344d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:44.800433 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49e264-4ba4-4507-bae5-70c49eb344d3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:44.825008 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49e264-4ba4-4507-bae5-70c49eb344d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:44.832695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-6b49e264-4ba4-4507-bae5-70c49eb344d3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:41:44.832695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:44.832695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:44.832695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:44.832695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:44.832695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:44.832695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:44.832695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:44.832695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:44.832695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:44.832695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:44.832695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:44.832695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:44.832695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:44.832695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:44.832695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:44.833626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:44.833626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:44.833626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:44.833626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:44.833626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:44.833626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:44.833626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:44.833626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:44.833626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:44.833626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:44.833626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:44.833626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:44.833626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:44.833626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:44.833626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:44.833626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:44.834409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:44.834409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:44.834409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:44.834409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:44.834409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:44.834409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:44.834409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:44.834409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:44.834409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:44.834409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:44.834409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:41:44.834409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:44.834409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 122/240] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:44.839691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49e264-4ba4-4507-bae5-70c49eb344d3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:44.911579 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba8ece-5dc5-4f1c-9703-ce5e26748991 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:44.982750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef217f-07d1-48a9-8f71-a5a67da20d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:44.983385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef217f-07d1-48a9-8f71-a5a67da20d31 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:44.983662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef217f-07d1-48a9-8f71-a5a67da20d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:44.983936 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef217f-07d1-48a9-8f71-a5a67da20d31 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:44.984727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef217f-07d1-48a9-8f71-a5a67da20d31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:44.986750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef217f-07d1-48a9-8f71-a5a67da20d31 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:45.016761 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49e264-4ba4-4507-bae5-70c49eb344d3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:45.020798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:45.039529 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 119/241] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:41:44 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:41:45.065719 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766d41e-5cb7-4de0-9604-ede66c22e4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:45.066352 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766d41e-5cb7-4de0-9604-ede66c22e4ce None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:45.066633 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766d41e-5cb7-4de0-9604-ede66c22e4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:45.066906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766d41e-5cb7-4de0-9604-ede66c22e4ce None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:45.067595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766d41e-5cb7-4de0-9604-ede66c22e4ce None admin] RBAC: Adding query filter params (name=reader) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:45.067898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766d41e-5cb7-4de0-9604-ede66c22e4ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:45.069806 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766d41e-5cb7-4de0-9604-ede66c22e4ce None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:45.079330 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 123/242] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:45.086047 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef217f-07d1-48a9-8f71-a5a67da20d31 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:45.168216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:45.226404 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef217f-07d1-48a9-8f71-a5a67da20d31 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:45.229241 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8e8dc-04da-4075-9545-fcd47fece52d None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:45.229519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8e8dc-04da-4075-9545-fcd47fece52d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:45.229793 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8e8dc-04da-4075-9545-fcd47fece52d None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:45.231367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 124/243] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:45.243280 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f64f07-27ba-4c3a-8ce6-44437336c0f6 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:45.243586 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f64f07-27ba-4c3a-8ce6-44437336c0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:45.243824 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f64f07-27ba-4c3a-8ce6-44437336c0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:45.273057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-47f64f07-27ba-4c3a-8ce6-44437336c0f6 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:45.274291 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-47f64f07-27ba-4c3a-8ce6-44437336c0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:45.293364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a819e-0195-46a9-9bb8-cd848350e7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:45.294003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a819e-0195-46a9-9bb8-cd848350e7fc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:45.294278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a819e-0195-46a9-9bb8-cd848350e7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:45.294548 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a819e-0195-46a9-9bb8-cd848350e7fc None admin] PATH_INFO: `/v3/users/63318198be344fbba10bc53d6c0c1bb9` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:45.310215 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a819e-0195-46a9-9bb8-cd848350e7fc None admin] RBAC: Authorizing `identity:get_user(user_id=63318198be344fbba10bc53d6c0c1bb9)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:45.311692 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a819e-0195-46a9-9bb8-cd848350e7fc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:45.313806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 120/244] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:45 2022] GET /identity/v3/users/63318198be344fbba10bc53d6c0c1bb9 => generated 297 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:45.321279 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a819e-0195-46a9-9bb8-cd848350e7fc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:45.359684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 125/245] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:45.427019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a819e-0195-46a9-9bb8-cd848350e7fc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:45.448443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:45.467527 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac78ec-580d-417e-97f4-04910ba47a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:45.468168 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac78ec-580d-417e-97f4-04910ba47a5c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:45.468450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac78ec-580d-417e-97f4-04910ba47a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:45.468721 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac78ec-580d-417e-97f4-04910ba47a5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:45.550004 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac78ec-580d-417e-97f4-04910ba47a5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:45.551544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac78ec-580d-417e-97f4-04910ba47a5c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:45.673033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 121/246] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:41:45.679584 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f64f07-27ba-4c3a-8ce6-44437336c0f6 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:45.702779 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac78ec-580d-417e-97f4-04910ba47a5c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:45.724787 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:45.807471 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:45.819871 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f64f07-27ba-4c3a-8ce6-44437336c0f6 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:45.845406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac78ec-580d-417e-97f4-04910ba47a5c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:45.858082 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b4140-1255-4e04-b5a5-58ae7131a58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:45.859077 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b4140-1255-4e04-b5a5-58ae7131a58b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:45.859524 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b4140-1255-4e04-b5a5-58ae7131a58b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:45.859963 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b4140-1255-4e04-b5a5-58ae7131a58b None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:45.861950 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b4140-1255-4e04-b5a5-58ae7131a58b None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:45.863712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b4140-1255-4e04-b5a5-58ae7131a58b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:45.865662 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 126/247] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:45 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:45.873972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b4140-1255-4e04-b5a5-58ae7131a58b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:45.895692 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ecfa1-a297-49be-9046-99826bda32c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:45.896737 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ecfa1-a297-49be-9046-99826bda32c1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:45.897185 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ecfa1-a297-49be-9046-99826bda32c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:45.897745 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ecfa1-a297-49be-9046-99826bda32c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:45.909604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ecfa1-a297-49be-9046-99826bda32c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:45.915690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-c45ecfa1-a297-49be-9046-99826bda32c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:45.915690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:45.915690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:45.915690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:45.915690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:45.915690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:45.915690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:45.915690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:45.915690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:45.915690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:45.915690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:45.915690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:45.915690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:45.915690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:45.915690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:45.915690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:45.916660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:45.916660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:45.916660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:45.916660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:45.916660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:45.916660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:45.916660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:45.916660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:45.916660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:45.916660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:45.916660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:45.916660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:45.916660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:45.916660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:45.916660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:45.916660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:45.917411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:45.917411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:45.917411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:45.917411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:45.917411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:45.917411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:45.917411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:45.917411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:45.917411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:45.917411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:45.917411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:45.917411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:45.924594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 122/248] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:41:45.927699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ecfa1-a297-49be-9046-99826bda32c1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:45.997366 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b4140-1255-4e04-b5a5-58ae7131a58b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:46.014213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:46.038719 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d1a91-34d1-45d0-9d6a-aa5c28c59188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:46.039333 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d1a91-34d1-45d0-9d6a-aa5c28c59188 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:46.039612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d1a91-34d1-45d0-9d6a-aa5c28c59188 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:46.040011 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d1a91-34d1-45d0-9d6a-aa5c28c59188 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:46.044980 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ecfa1-a297-49be-9046-99826bda32c1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:46.046325 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d1a91-34d1-45d0-9d6a-aa5c28c59188 None admin] RBAC: Adding query filter params (role.id=a5350d487c374ea680a13ac3044168fc, scope.project.id=ea579b172f214888a95d9af5a4e1d122, user.id=63318198be344fbba10bc53d6c0c1bb9) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:46.046444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d1a91-34d1-45d0-9d6a-aa5c28c59188 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:46.047872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d1a91-34d1-45d0-9d6a-aa5c28c59188 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:46.055218 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 127/249] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:45 2022] GET /identity/v3/role_assignments?user.id=63318198be344fbba10bc53d6c0c1bb9&scope.project.id=ea579b172f214888a95d9af5a4e1d122&role.id=a5350d487c374ea680a13ac3044168fc => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:46.077027 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816a163-19bd-46c4-b445-da868c2d070e None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:46.077310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816a163-19bd-46c4-b445-da868c2d070e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:46.077589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816a163-19bd-46c4-b445-da868c2d070e None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:46.079416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 128/250] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:41:46.086181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab7e29-17ec-4c22-8b58-3ba21ed69f2b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:46.086465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab7e29-17ec-4c22-8b58-3ba21ed69f2b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:46.086742 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab7e29-17ec-4c22-8b58-3ba21ed69f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:46.088885 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40941536-e53a-49a5-ad85-206e404f1286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:46.089675 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40941536-e53a-49a5-ad85-206e404f1286 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:46.090248 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40941536-e53a-49a5-ad85-206e404f1286 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:46.090517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40941536-e53a-49a5-ad85-206e404f1286 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:46.091237 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40941536-e53a-49a5-ad85-206e404f1286 None admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:46.091593 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40941536-e53a-49a5-ad85-206e404f1286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:46.093127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40941536-e53a-49a5-ad85-206e404f1286 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:46.104162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 123/251] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:46.114710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40941536-e53a-49a5-ad85-206e404f1286 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:46.128138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3eab7e29-17ec-4c22-8b58-3ba21ed69f2b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:46.129419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3eab7e29-17ec-4c22-8b58-3ba21ed69f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:46.182405 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:46.269908 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 129/252] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:46.319807 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40941536-e53a-49a5-ad85-206e404f1286 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:46.372766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fabb74-5bc5-4af9-82e5-d2bf3bfb96b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:46.373778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fabb74-5bc5-4af9-82e5-d2bf3bfb96b5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:46.374257 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fabb74-5bc5-4af9-82e5-d2bf3bfb96b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:46.374755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fabb74-5bc5-4af9-82e5-d2bf3bfb96b5 None admin] PATH_INFO: `/v3/users/b83b7b6bbea849d4b526b03c27564c2f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:46.376193 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fabb74-5bc5-4af9-82e5-d2bf3bfb96b5 None admin] RBAC: Authorizing `identity:get_user(user_id=b83b7b6bbea849d4b526b03c27564c2f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:46.378438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fabb74-5bc5-4af9-82e5-d2bf3bfb96b5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:46.380346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 124/253] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:46 2022] GET /identity/v3/users/b83b7b6bbea849d4b526b03c27564c2f => generated 297 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:46.389463 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab7e29-17ec-4c22-8b58-3ba21ed69f2b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:46.458624 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb44ccd-d938-46af-9d1c-fe3a71ddbf68 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:46.459256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb44ccd-d938-46af-9d1c-fe3a71ddbf68 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:46.459733 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb44ccd-d938-46af-9d1c-fe3a71ddbf68 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:46.461299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 125/254] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:41:46.478874 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361681c-c696-4c44-858e-f82f60271533 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:46.479449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361681c-c696-4c44-858e-f82f60271533 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:46.479924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361681c-c696-4c44-858e-f82f60271533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:46.491491 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:46.507105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d361681c-c696-4c44-858e-f82f60271533 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:46.508599 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d361681c-c696-4c44-858e-f82f60271533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:46.563644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab7e29-17ec-4c22-8b58-3ba21ed69f2b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:46.629545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 126/255] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 07 07:41:46.633851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510430e-5702-423f-92f3-1142b1c80b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:46.634574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510430e-5702-423f-92f3-1142b1c80b75 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:46.634963 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510430e-5702-423f-92f3-1142b1c80b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:46.635316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510430e-5702-423f-92f3-1142b1c80b75 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:46.636956 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510430e-5702-423f-92f3-1142b1c80b75 None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:46.638484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510430e-5702-423f-92f3-1142b1c80b75 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:46.640516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 130/256] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:46 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:46.646249 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361681c-c696-4c44-858e-f82f60271533 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:46.646493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510430e-5702-423f-92f3-1142b1c80b75 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:46.704442 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:46.799983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361681c-c696-4c44-858e-f82f60271533 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:46.833220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510430e-5702-423f-92f3-1142b1c80b75 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:46.837057 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e360c-91d9-4d2b-b169-d76cb65b4fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:46.837989 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e360c-91d9-4d2b-b169-d76cb65b4fb6 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:46.838469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e360c-91d9-4d2b-b169-d76cb65b4fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:46.838880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e360c-91d9-4d2b-b169-d76cb65b4fb6 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122/users/b83b7b6bbea849d4b526b03c27564c2f/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:46.841172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e360c-91d9-4d2b-b169-d76cb65b4fb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea579b172f214888a95d9af5a4e1d122, user_id=b83b7b6bbea849d4b526b03c27564c2f, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:46.842788 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e360c-91d9-4d2b-b169-d76cb65b4fb6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:46.845487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:46.866928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 127/257] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:41:46 2022] PUT /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122/users/b83b7b6bbea849d4b526b03c27564c2f/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:41:46.898162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d78d2d-62f5-4b29-b1c9-f4ef29f89a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:46.899050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d78d2d-62f5-4b29-b1c9-f4ef29f89a0c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:46.899467 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d78d2d-62f5-4b29-b1c9-f4ef29f89a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:46.899905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d78d2d-62f5-4b29-b1c9-f4ef29f89a0c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:46.911778 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d78d2d-62f5-4b29-b1c9-f4ef29f89a0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:46.917131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-99d78d2d-62f5-4b29-b1c9-f4ef29f89a0c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:41:46.917131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:46.917131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:46.917131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:46.917131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:46.917131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:46.917131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:46.917131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:46.917131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:46.917131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:46.917131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:46.917131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:46.917131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:46.917131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:46.917131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:46.917131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:46.918316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:46.918316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:46.918316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:46.918316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:46.918316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:46.918316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:46.918316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:46.918316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:46.918316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:46.918316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:46.918316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:46.918316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:46.918316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:46.918316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:46.918316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:46.918316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:46.919285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:46.919285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:46.919285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:46.919285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:46.919285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:46.919285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:46.919285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:46.919285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:46.919285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:46.919285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:46.919285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:41:46.919285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:46.921281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 131/258] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:41:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:41:46.928206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e360c-91d9-4d2b-b169-d76cb65b4fb6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:47.041933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e360c-91d9-4d2b-b169-d76cb65b4fb6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:47.080525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4aafc0-62a8-418e-b62d-b25b5a8bdf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:47.081804 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aafc0-62a8-418e-b62d-b25b5a8bdf89 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:47.082222 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aafc0-62a8-418e-b62d-b25b5a8bdf89 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:47.082654 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aafc0-62a8-418e-b62d-b25b5a8bdf89 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:47.083546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aafc0-62a8-418e-b62d-b25b5a8bdf89 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:47.083996 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aafc0-62a8-418e-b62d-b25b5a8bdf89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:47.086031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d78d2d-62f5-4b29-b1c9-f4ef29f89a0c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:47.089007 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aafc0-62a8-418e-b62d-b25b5a8bdf89 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:47.103360 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 128/259] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:41:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:47.107278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4aafc0-62a8-418e-b62d-b25b5a8bdf89 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:47.219880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:47.268653 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:47.294557 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4aafc0-62a8-418e-b62d-b25b5a8bdf89 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:47.296268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d78d2d-62f5-4b29-b1c9-f4ef29f89a0c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:47.336181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2deb1ba-28a6-4e7b-96b7-2ed5b6cac50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:47.336181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deb1ba-28a6-4e7b-96b7-2ed5b6cac50e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:47.336181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deb1ba-28a6-4e7b-96b7-2ed5b6cac50e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:47.336181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deb1ba-28a6-4e7b-96b7-2ed5b6cac50e None admin] PATH_INFO: `/v3/users/db5dc2d21805484b9ef54fb44f746191` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:47.336540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2deb1ba-28a6-4e7b-96b7-2ed5b6cac50e None admin] RBAC: Authorizing `identity:get_user(user_id=db5dc2d21805484b9ef54fb44f746191)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:47.336540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2deb1ba-28a6-4e7b-96b7-2ed5b6cac50e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:47.341282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 129/260] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:47 2022] GET /identity/v3/users/db5dc2d21805484b9ef54fb44f746191 => generated 283 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:47.345016 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2deb1ba-28a6-4e7b-96b7-2ed5b6cac50e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:47.373371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc863f-8d61-4c8c-b9dc-3cb21e2e637d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:47.374017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc863f-8d61-4c8c-b9dc-3cb21e2e637d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:47.374295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc863f-8d61-4c8c-b9dc-3cb21e2e637d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:47.374565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc863f-8d61-4c8c-b9dc-3cb21e2e637d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:47.397951 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc863f-8d61-4c8c-b9dc-3cb21e2e637d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:47.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-8bfc863f-8d61-4c8c-b9dc-3cb21e2e637d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:47.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:47.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:47.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:47.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:47.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:47.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:47.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:47.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:47.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:47.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:47.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:47.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:47.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:47.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:47.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:47.407946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:47.407946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:47.407946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:47.407946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:47.407946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:47.407946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:47.407946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:47.407946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:47.407946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:47.407946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:47.407946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:47.407946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:47.407946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:47.407946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:47.407946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:47.407946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:47.408868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:47.408868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:47.408868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:47.408868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:47.408868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:47.408868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:47.408868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:47.408868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:47.408868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:47.408868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:47.408868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:41:47.408868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:47.408868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 132/261] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:41:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:41:47.425173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc863f-8d61-4c8c-b9dc-3cb21e2e637d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:47.472236 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2deb1ba-28a6-4e7b-96b7-2ed5b6cac50e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:47.500492 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:47.509023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f176c65-0798-4c3d-b284-27643c69c26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:47.509632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f176c65-0798-4c3d-b284-27643c69c26b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:47.509912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f176c65-0798-4c3d-b284-27643c69c26b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:47.510188 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f176c65-0798-4c3d-b284-27643c69c26b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:47.607961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f176c65-0798-4c3d-b284-27643c69c26b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:47.613339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f176c65-0798-4c3d-b284-27643c69c26b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:47.676163 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc863f-8d61-4c8c-b9dc-3cb21e2e637d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:47.758834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6d310-1fb4-467f-a452-3e8600f46e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:47.759804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6d310-1fb4-467f-a452-3e8600f46e47 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:47.760311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6d310-1fb4-467f-a452-3e8600f46e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:47.760812 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6d310-1fb4-467f-a452-3e8600f46e47 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:47.761732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6d310-1fb4-467f-a452-3e8600f46e47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:47.762234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6d310-1fb4-467f-a452-3e8600f46e47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:47.763829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6d310-1fb4-467f-a452-3e8600f46e47 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:47.786626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 130/262] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 07:41:47.788039 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f176c65-0798-4c3d-b284-27643c69c26b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:47.794238 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 133/263] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:47.800551 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6d310-1fb4-467f-a452-3e8600f46e47 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:47.812977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:47.824433 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:47.926839 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f176c65-0798-4c3d-b284-27643c69c26b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:47.981980 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96595b-4ff0-49fa-bd48-c3cdfe41d677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:47.982610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96595b-4ff0-49fa-bd48-c3cdfe41d677 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:47.982883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96595b-4ff0-49fa-bd48-c3cdfe41d677 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:47.983151 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96595b-4ff0-49fa-bd48-c3cdfe41d677 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:47.984304 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96595b-4ff0-49fa-bd48-c3cdfe41d677 None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:47.985747 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96595b-4ff0-49fa-bd48-c3cdfe41d677 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:47.988778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 131/264] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:47 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:47.993650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96595b-4ff0-49fa-bd48-c3cdfe41d677 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:48.041134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6d310-1fb4-467f-a452-3e8600f46e47 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:48.086117 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616cfae-cb09-4e8d-8231-dd976669512e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:48.086766 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616cfae-cb09-4e8d-8231-dd976669512e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:48.087317 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616cfae-cb09-4e8d-8231-dd976669512e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:48.087589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616cfae-cb09-4e8d-8231-dd976669512e None admin] PATH_INFO: `/v3/users/db5dc2d21805484b9ef54fb44f746191` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:48.090083 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616cfae-cb09-4e8d-8231-dd976669512e None admin] RBAC: Authorizing `identity:get_user(user_id=db5dc2d21805484b9ef54fb44f746191)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:48.092514 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616cfae-cb09-4e8d-8231-dd976669512e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:48.100577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 134/265] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:47 2022] GET /identity/v3/users/db5dc2d21805484b9ef54fb44f746191 => generated 283 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:48.103698 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616cfae-cb09-4e8d-8231-dd976669512e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:48.120639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:48.149243 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96595b-4ff0-49fa-bd48-c3cdfe41d677 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:48.233654 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3fd64-ebf8-4b89-a3ed-a165954583fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:48.234181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3fd64-ebf8-4b89-a3ed-a165954583fa None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:48.234454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3fd64-ebf8-4b89-a3ed-a165954583fa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:48.234719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3fd64-ebf8-4b89-a3ed-a165954583fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:48.235502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3fd64-ebf8-4b89-a3ed-a165954583fa None admin] RBAC: Adding query filter params (role.id=84a30faea3e94eb48b03ec7ceb7c1392, scope.project.id=ea579b172f214888a95d9af5a4e1d122, user.id=db5dc2d21805484b9ef54fb44f746191) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:48.235807 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3fd64-ebf8-4b89-a3ed-a165954583fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:48.245582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3fd64-ebf8-4b89-a3ed-a165954583fa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:48.256503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 132/266] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:47 2022] GET /identity/v3/role_assignments?user.id=db5dc2d21805484b9ef54fb44f746191&scope.project.id=ea579b172f214888a95d9af5a4e1d122&role.id=84a30faea3e94eb48b03ec7ceb7c1392 => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:48.284534 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:48.316482 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616cfae-cb09-4e8d-8231-dd976669512e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:48.371616 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba74e25-a32b-4cec-81b4-a886456d4ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:48.372279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba74e25-a32b-4cec-81b4-a886456d4ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:48.372561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba74e25-a32b-4cec-81b4-a886456d4ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:48.372828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba74e25-a32b-4cec-81b4-a886456d4ea3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:48.460579 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba74e25-a32b-4cec-81b4-a886456d4ea3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:48.462065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba74e25-a32b-4cec-81b4-a886456d4ea3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:48.545551 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:48.571440 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 135/267] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:41:48.577377 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3fd64-ebf8-4b89-a3ed-a165954583fa None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:48.701304 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a41579-dd65-48c5-afe5-38dfe8ca4e90 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:48.701622 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a41579-dd65-48c5-afe5-38dfe8ca4e90 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:48.701856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a41579-dd65-48c5-afe5-38dfe8ca4e90 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:48.703196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 136/268] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:41:48.724685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3fd64-ebf8-4b89-a3ed-a165954583fa None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:48.725540 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e6bd7-90c1-43f6-a0b8-8855997ccbc3 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:48.726054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e6bd7-90c1-43f6-a0b8-8855997ccbc3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:48.726545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e6bd7-90c1-43f6-a0b8-8855997ccbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:48.764734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90009d7e-0b5c-45e4-b266-a9a904796356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:48.765363 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90009d7e-0b5c-45e4-b266-a9a904796356 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:48.765638 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90009d7e-0b5c-45e4-b266-a9a904796356 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:48.765896 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b47e6bd7-90c1-43f6-a0b8-8855997ccbc3 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:48.766011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90009d7e-0b5c-45e4-b266-a9a904796356 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:48.767032 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90009d7e-0b5c-45e4-b266-a9a904796356 None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:48.768168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b47e6bd7-90c1-43f6-a0b8-8855997ccbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:48.769298 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90009d7e-0b5c-45e4-b266-a9a904796356 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:48.771288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 133/269] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:48 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:48.776741 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90009d7e-0b5c-45e4-b266-a9a904796356 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:48.807680 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:48.886361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90009d7e-0b5c-45e4-b266-a9a904796356 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:48.909578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 137/270] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:48.924192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655e764-cb3b-4d7e-982a-3b95f0fc5810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:48.925431 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655e764-cb3b-4d7e-982a-3b95f0fc5810 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:48.925907 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655e764-cb3b-4d7e-982a-3b95f0fc5810 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:48.926367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655e764-cb3b-4d7e-982a-3b95f0fc5810 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:48.927294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655e764-cb3b-4d7e-982a-3b95f0fc5810 None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=ea579b172f214888a95d9af5a4e1d122, user.id=db5dc2d21805484b9ef54fb44f746191) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:48.927863 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655e764-cb3b-4d7e-982a-3b95f0fc5810 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:48.929735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655e764-cb3b-4d7e-982a-3b95f0fc5810 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:48.936829 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 134/271] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:48 2022] GET /identity/v3/role_assignments?user.id=db5dc2d21805484b9ef54fb44f746191&scope.project.id=ea579b172f214888a95d9af5a4e1d122&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:49.226791 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e6bd7-90c1-43f6-a0b8-8855997ccbc3 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:49.304664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dce37-651f-45ad-ae79-2aeed9f914f2 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:49.305609 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dce37-651f-45ad-ae79-2aeed9f914f2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:49.306099 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dce37-651f-45ad-ae79-2aeed9f914f2 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:49.309157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 135/272] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:49.314576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95d97d-5ed3-4e2e-ab20-343720fa777d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:49.315183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95d97d-5ed3-4e2e-ab20-343720fa777d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:49.315781 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95d97d-5ed3-4e2e-ab20-343720fa777d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:49.339097 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6a95d97d-5ed3-4e2e-ab20-343720fa777d None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:49.340754 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6a95d97d-5ed3-4e2e-ab20-343720fa777d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:49.344525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e6bd7-90c1-43f6-a0b8-8855997ccbc3 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:49.352465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:49.402195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a5b79-7ce4-446f-8c85-114cfa65c2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:49.402195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a5b79-7ce4-446f-8c85-114cfa65c2eb None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:49.402195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a5b79-7ce4-446f-8c85-114cfa65c2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:49.402195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a5b79-7ce4-446f-8c85-114cfa65c2eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:49.421012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a5b79-7ce4-446f-8c85-114cfa65c2eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:49.427763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-bb7a5b79-7ce4-446f-8c85-114cfa65c2eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:49.427763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:49.427763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:49.427763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:49.427763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:49.427763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:49.427763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:49.427763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:49.427763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:49.427763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:49.427763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:49.427763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:49.427763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:49.427763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:49.427763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:49.427763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:49.428787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:49.428787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:49.428787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:49.428787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:49.428787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:49.428787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:49.428787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:49.428787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:49.428787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:49.428787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:49.428787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:49.428787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:49.428787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:49.428787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:49.428787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:49.428787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:49.429564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:49.429564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:49.429564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:49.429564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:49.429564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:49.429564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:49.429564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:49.429564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:49.429564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:49.429564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:49.429564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:49.429564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:49.429564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 138/273] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:49.436244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a5b79-7ce4-446f-8c85-114cfa65c2eb None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:49.450143 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 136/274] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:49.549222 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb86bfc-cf7e-411c-8ea7-453199bb41bf None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:49.549530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb86bfc-cf7e-411c-8ea7-453199bb41bf None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:49.549670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb86bfc-cf7e-411c-8ea7-453199bb41bf None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:49.551293 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 137/275] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:41:49.554571 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a5b79-7ce4-446f-8c85-114cfa65c2eb None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:49.556462 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52def768-bd10-44f2-b3df-7bb43c6207a2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:49.556750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52def768-bd10-44f2-b3df-7bb43c6207a2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:49.557023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52def768-bd10-44f2-b3df-7bb43c6207a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:49.575828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-52def768-bd10-44f2-b3df-7bb43c6207a2 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:49.577060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-52def768-bd10-44f2-b3df-7bb43c6207a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:49.597973 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:49.600776 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcfac9-80d4-4332-a945-0f2d5f11f25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:49.601667 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcfac9-80d4-4332-a945-0f2d5f11f25e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:49.601944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcfac9-80d4-4332-a945-0f2d5f11f25e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:49.602218 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcfac9-80d4-4332-a945-0f2d5f11f25e None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:49.602884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcfac9-80d4-4332-a945-0f2d5f11f25e None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:49.603187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcfac9-80d4-4332-a945-0f2d5f11f25e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:49.604613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcfac9-80d4-4332-a945-0f2d5f11f25e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:49.614639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 139/276] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:49.620358 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcfac9-80d4-4332-a945-0f2d5f11f25e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:49.653146 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:49.675255 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 138/277] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:49.738521 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcfac9-80d4-4332-a945-0f2d5f11f25e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:49.749060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52def768-bd10-44f2-b3df-7bb43c6207a2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:49.776148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f361a70-c03b-4062-9852-3a5c2fa14901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:49.776783 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f361a70-c03b-4062-9852-3a5c2fa14901 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:49.777057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f361a70-c03b-4062-9852-3a5c2fa14901 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:49.777328 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f361a70-c03b-4062-9852-3a5c2fa14901 None admin] PATH_INFO: `/v3/users/621389d663684029978592fbf244d1d4` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:49.791980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f361a70-c03b-4062-9852-3a5c2fa14901 None admin] RBAC: Authorizing `identity:get_user(user_id=621389d663684029978592fbf244d1d4)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:49.793531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f361a70-c03b-4062-9852-3a5c2fa14901 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:49.795544 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 140/278] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:49 2022] GET /identity/v3/users/621389d663684029978592fbf244d1d4 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:49.807353 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f361a70-c03b-4062-9852-3a5c2fa14901 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:49.874556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52def768-bd10-44f2-b3df-7bb43c6207a2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:49.902258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:49.913396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a466715-a535-4181-be5e-e6c3e8117b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:49.914381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a466715-a535-4181-be5e-e6c3e8117b10 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:49.915117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a466715-a535-4181-be5e-e6c3e8117b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:49.915866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a466715-a535-4181-be5e-e6c3e8117b10 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:49.920770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f361a70-c03b-4062-9852-3a5c2fa14901 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:49.929030 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a466715-a535-4181-be5e-e6c3e8117b10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:49.935578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-7a466715-a535-4181-be5e-e6c3e8117b10 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:41:49.935578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:49.935578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:49.935578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:49.935578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:49.935578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:49.935578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:49.935578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:49.935578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:49.935578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:49.935578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:49.935578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:49.935578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:49.935578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:49.935578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:49.935578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:49.936653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:49.936653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:49.936653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:49.936653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:49.936653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:49.936653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:49.936653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:49.936653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:49.936653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:49.936653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:49.936653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:49.936653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:49.936653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:49.936653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:49.936653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:49.936653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:49.937636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:49.937636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:49.937636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:49.937636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:49.937636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:49.937636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:49.937636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:49.937636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:49.937636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:49.937636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:49.937636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:41:49.937636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:49.940881 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 139/279] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:49.942550 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce1753-e13b-4728-a781-1e4152ba29a8 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:49.943092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce1753-e13b-4728-a781-1e4152ba29a8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:49.943765 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce1753-e13b-4728-a781-1e4152ba29a8 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:49.945999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 140/280] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:49.947243 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce1753-e13b-4728-a781-1e4152ba29a8 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:49.961508 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:49.962539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ee8f4-bc94-4baa-a59f-1d8a0b47f833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:49.963156 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ee8f4-bc94-4baa-a59f-1d8a0b47f833 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:49.963463 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ee8f4-bc94-4baa-a59f-1d8a0b47f833 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:49.963808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ee8f4-bc94-4baa-a59f-1d8a0b47f833 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:49.964640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ee8f4-bc94-4baa-a59f-1d8a0b47f833 None admin] RBAC: Adding query filter params (role.id=6ebbed85905048a198e4261a3510dafc, user.id=621389d663684029978592fbf244d1d4, scope.system=all) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:49.964954 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ee8f4-bc94-4baa-a59f-1d8a0b47f833 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:49.966430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ee8f4-bc94-4baa-a59f-1d8a0b47f833 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:49.971469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 141/281] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Sep 7 07:41:49 2022] GET /identity/v3/role_assignments?user.id=621389d663684029978592fbf244d1d4&scope.system=all&role.id=6ebbed85905048a198e4261a3510dafc => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:49.972694 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f80fd9-6b82-4f09-84e6-7447011f17a0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:49.972984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f80fd9-6b82-4f09-84e6-7447011f17a0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:49.973258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f80fd9-6b82-4f09-84e6-7447011f17a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:49.993310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e0f80fd9-6b82-4f09-84e6-7447011f17a0 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:49.994651 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e0f80fd9-6b82-4f09-84e6-7447011f17a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:50.075498 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce1753-e13b-4728-a781-1e4152ba29a8 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:50.076690 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 142/282] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:50.080984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f80fd9-6b82-4f09-84e6-7447011f17a0 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:50.125554 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b401f5-20be-46d8-b30c-0f441f3109f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:50.126277 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b401f5-20be-46d8-b30c-0f441f3109f0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:50.126626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b401f5-20be-46d8-b30c-0f441f3109f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:50.127457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b401f5-20be-46d8-b30c-0f441f3109f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:50.128311 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b401f5-20be-46d8-b30c-0f441f3109f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:50.128835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b401f5-20be-46d8-b30c-0f441f3109f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:50.130391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b401f5-20be-46d8-b30c-0f441f3109f0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:50.140242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 141/283] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:50.149152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b401f5-20be-46d8-b30c-0f441f3109f0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:50.182941 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:50.196476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f80fd9-6b82-4f09-84e6-7447011f17a0 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:50.231508 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c8df0-7d10-4277-87a7-eea70b1b4973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:50.232931 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c8df0-7d10-4277-87a7-eea70b1b4973 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:50.233426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c8df0-7d10-4277-87a7-eea70b1b4973 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:50.233888 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c8df0-7d10-4277-87a7-eea70b1b4973 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:50.235217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c8df0-7d10-4277-87a7-eea70b1b4973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:50.237425 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c8df0-7d10-4277-87a7-eea70b1b4973 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:50.239268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 143/284] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:50.244909 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c8df0-7d10-4277-87a7-eea70b1b4973 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:50.250105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:50.274306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b401f5-20be-46d8-b30c-0f441f3109f0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:50.309299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0f3be-017f-41d4-8115-689719099ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:50.310061 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0f3be-017f-41d4-8115-689719099ecf None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:50.310227 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0f3be-017f-41d4-8115-689719099ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:50.310479 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0f3be-017f-41d4-8115-689719099ecf None admin] PATH_INFO: `/v3/users/63318198be344fbba10bc53d6c0c1bb9` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:50.311631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0f3be-017f-41d4-8115-689719099ecf None admin] RBAC: Authorizing `identity:get_user(user_id=63318198be344fbba10bc53d6c0c1bb9)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:50.313102 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0f3be-017f-41d4-8115-689719099ecf None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:50.319617 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 142/285] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:50 2022] GET /identity/v3/users/63318198be344fbba10bc53d6c0c1bb9 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:50.320529 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0f3be-017f-41d4-8115-689719099ecf None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:50.354919 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c8df0-7d10-4277-87a7-eea70b1b4973 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:50.395342 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3d8b2-5d37-4788-ba49-6f2907446df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:50.395953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3d8b2-5d37-4788-ba49-6f2907446df7 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:50.396270 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3d8b2-5d37-4788-ba49-6f2907446df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:50.396542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3d8b2-5d37-4788-ba49-6f2907446df7 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:50.397191 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3d8b2-5d37-4788-ba49-6f2907446df7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:50.398643 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3d8b2-5d37-4788-ba49-6f2907446df7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:50.430319 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 144/286] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:41:50 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:41:50.431635 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3d8b2-5d37-4788-ba49-6f2907446df7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:50.442372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:50.450080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0f3be-017f-41d4-8115-689719099ecf None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:50.498891 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7472a-a5c0-48ed-8dee-fadf7c5a1ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:50.499963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7472a-a5c0-48ed-8dee-fadf7c5a1ace None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:50.500293 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7472a-a5c0-48ed-8dee-fadf7c5a1ace None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:50.500573 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7472a-a5c0-48ed-8dee-fadf7c5a1ace None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:50.502661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7472a-a5c0-48ed-8dee-fadf7c5a1ace None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:50.504477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7472a-a5c0-48ed-8dee-fadf7c5a1ace None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:50.507187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 143/287] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:50 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:50.513740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7472a-a5c0-48ed-8dee-fadf7c5a1ace None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:50.527449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:50.568405 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3d8b2-5d37-4788-ba49-6f2907446df7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:50.607751 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e1003-81a1-42ee-9b61-d9fa453eb531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:50.608419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e1003-81a1-42ee-9b61-d9fa453eb531 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:50.608781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e1003-81a1-42ee-9b61-d9fa453eb531 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:50.609123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e1003-81a1-42ee-9b61-d9fa453eb531 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:50.628234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e1003-81a1-42ee-9b61-d9fa453eb531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:50.633644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-669e1003-81a1-42ee-9b61-d9fa453eb531 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:50.633644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:50.633644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:50.633644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:50.633644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:50.633644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:50.633644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:50.633644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:50.633644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:50.633644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:50.633644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:50.633644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:50.633644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:50.633644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:50.633644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:50.633644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:50.634495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:50.634495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:50.634495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:50.634495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:50.634495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:50.634495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:50.634495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:50.634495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:50.634495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:50.634495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:50.634495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:50.634495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:50.634495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:50.634495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:50.634495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:50.634495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:50.635295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:50.635295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:50.635295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:50.635295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:50.635295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:50.635295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:50.635295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:50.635295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:50.635295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:50.635295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:50.635295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:50.635295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:50.635295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 145/288] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:50.641378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e1003-81a1-42ee-9b61-d9fa453eb531 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:50.650658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7472a-a5c0-48ed-8dee-fadf7c5a1ace None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:50.697692 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67507d45-6607-443c-bb3f-f86f2186f400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:50.698365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67507d45-6607-443c-bb3f-f86f2186f400 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:50.698644 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67507d45-6607-443c-bb3f-f86f2186f400 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:50.698914 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67507d45-6607-443c-bb3f-f86f2186f400 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122/users/63318198be344fbba10bc53d6c0c1bb9/roles/a5350d487c374ea680a13ac3044168fc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:50.701209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67507d45-6607-443c-bb3f-f86f2186f400 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea579b172f214888a95d9af5a4e1d122, user_id=63318198be344fbba10bc53d6c0c1bb9, role_id=a5350d487c374ea680a13ac3044168fc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:50.702335 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67507d45-6607-443c-bb3f-f86f2186f400 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:50.719097 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 144/289] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:41:50 2022] PUT /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122/users/63318198be344fbba10bc53d6c0c1bb9/roles/a5350d487c374ea680a13ac3044168fc => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:41:50.731197 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:50.785780 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e1003-81a1-42ee-9b61-d9fa453eb531 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:50.826365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117accf3-ae09-44d1-9268-1ed06216883c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:50.826991 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117accf3-ae09-44d1-9268-1ed06216883c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:50.827266 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117accf3-ae09-44d1-9268-1ed06216883c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:50.827534 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117accf3-ae09-44d1-9268-1ed06216883c None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:50.828238 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117accf3-ae09-44d1-9268-1ed06216883c None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:50.828538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117accf3-ae09-44d1-9268-1ed06216883c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:50.829957 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117accf3-ae09-44d1-9268-1ed06216883c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:50.841020 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 146/290] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:50.847781 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67507d45-6607-443c-bb3f-f86f2186f400 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:50.945501 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:50.985515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67507d45-6607-443c-bb3f-f86f2186f400 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:51.020310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a60c4-1787-46d8-a07a-5cca5fc06ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:51.020944 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a60c4-1787-46d8-a07a-5cca5fc06ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:51.021223 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a60c4-1787-46d8-a07a-5cca5fc06ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:51.021494 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a60c4-1787-46d8-a07a-5cca5fc06ea7 None admin] PATH_INFO: `/v3/users/b83b7b6bbea849d4b526b03c27564c2f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:51.022641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a60c4-1787-46d8-a07a-5cca5fc06ea7 None admin] RBAC: Authorizing `identity:get_user(user_id=b83b7b6bbea849d4b526b03c27564c2f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:51.024105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a60c4-1787-46d8-a07a-5cca5fc06ea7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:51.026056 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 145/291] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:50 2022] GET /identity/v3/users/b83b7b6bbea849d4b526b03c27564c2f => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:51.032446 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117accf3-ae09-44d1-9268-1ed06216883c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:51.178406 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117accf3-ae09-44d1-9268-1ed06216883c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:51.214066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:51.227443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc3d55-985e-4b44-9fe1-5a27476a04d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:51.228269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc3d55-985e-4b44-9fe1-5a27476a04d5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:51.228686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc3d55-985e-4b44-9fe1-5a27476a04d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:51.229117 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc3d55-985e-4b44-9fe1-5a27476a04d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:51.319254 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc3d55-985e-4b44-9fe1-5a27476a04d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:51.320771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc3d55-985e-4b44-9fe1-5a27476a04d5 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:51.449957 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 147/292] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:41:51.455546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a60c4-1787-46d8-a07a-5cca5fc06ea7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:51.586334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a60c4-1787-46d8-a07a-5cca5fc06ea7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:51.620231 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d6ada-f087-4521-a7d5-cf94873e14a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:51.620865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d6ada-f087-4521-a7d5-cf94873e14a2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:51.621140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d6ada-f087-4521-a7d5-cf94873e14a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:51.621415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d6ada-f087-4521-a7d5-cf94873e14a2 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:51.622609 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d6ada-f087-4521-a7d5-cf94873e14a2 None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:51.624060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d6ada-f087-4521-a7d5-cf94873e14a2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:51.626041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 146/293] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:51 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:51.633168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc3d55-985e-4b44-9fe1-5a27476a04d5 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:51.669798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:51.779892 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc3d55-985e-4b44-9fe1-5a27476a04d5 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:51.829309 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ae4ef-f517-4066-b573-42921097875a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:51.829969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ae4ef-f517-4066-b573-42921097875a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:51.830342 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ae4ef-f517-4066-b573-42921097875a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:51.830812 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ae4ef-f517-4066-b573-42921097875a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:51.832217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ae4ef-f517-4066-b573-42921097875a None admin] RBAC: Adding query filter params (role.id=6ebbed85905048a198e4261a3510dafc, scope.project.id=ea579b172f214888a95d9af5a4e1d122, user.id=b83b7b6bbea849d4b526b03c27564c2f) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:51.832543 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ae4ef-f517-4066-b573-42921097875a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:51.833989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ae4ef-f517-4066-b573-42921097875a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:51.844945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 148/294] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:51 2022] GET /identity/v3/role_assignments?user.id=b83b7b6bbea849d4b526b03c27564c2f&scope.project.id=ea579b172f214888a95d9af5a4e1d122&role.id=6ebbed85905048a198e4261a3510dafc => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:52.471993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c7cae-e3e0-4cd9-8c77-14ba91eb6cc0 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:52.472543 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c7cae-e3e0-4cd9-8c77-14ba91eb6cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:52.472543 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c7cae-e3e0-4cd9-8c77-14ba91eb6cc0 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:52.474199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 147/295] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:52.480149 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783016c7-b5eb-4367-a014-bc168dd59f1e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:52.480529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783016c7-b5eb-4367-a014-bc168dd59f1e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:52.480709 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783016c7-b5eb-4367-a014-bc168dd59f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:52.501092 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-783016c7-b5eb-4367-a014-bc168dd59f1e None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:52.502647 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-783016c7-b5eb-4367-a014-bc168dd59f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:52.596232 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 149/296] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:52.874840 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c7cae-e3e0-4cd9-8c77-14ba91eb6cc0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:52.878108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:52.987913 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c7cae-e3e0-4cd9-8c77-14ba91eb6cc0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:53.027292 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927d24c-4e69-42d7-a313-dae657632d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:53.027921 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927d24c-4e69-42d7-a313-dae657632d35 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:53.028224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927d24c-4e69-42d7-a313-dae657632d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:53.028497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927d24c-4e69-42d7-a313-dae657632d35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:53.039256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b927d24c-4e69-42d7-a313-dae657632d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:53.044647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-b927d24c-4e69-42d7-a313-dae657632d35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:53.044647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:53.044647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:53.044647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:53.044647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:53.044647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:53.044647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:53.044647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:53.044647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:53.044647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:53.044647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:53.044647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:53.044647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:53.044647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:53.044647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:53.044647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:53.045516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:53.045516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:53.045516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:53.045516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:53.045516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:53.045516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:53.045516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:53.045516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:53.045516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:53.045516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:53.045516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:53.045516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:53.045516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:53.045516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:53.045516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:53.045516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:53.046278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:53.046278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:53.046278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:53.046278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:53.046278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:53.046278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:53.046278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:53.046278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:53.046278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:53.046278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:53.046278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:53.046278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:53.046278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 148/297] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:53.052396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783016c7-b5eb-4367-a014-bc168dd59f1e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:53.075984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:53.161295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783016c7-b5eb-4367-a014-bc168dd59f1e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:53.201463 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a9dd6-7e67-47f6-b3a3-98b50b06182d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:53.202082 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a9dd6-7e67-47f6-b3a3-98b50b06182d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:53.202357 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a9dd6-7e67-47f6-b3a3-98b50b06182d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:53.202626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a9dd6-7e67-47f6-b3a3-98b50b06182d None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:53.203293 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0a9dd6-7e67-47f6-b3a3-98b50b06182d None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:53.203589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0a9dd6-7e67-47f6-b3a3-98b50b06182d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:53.205286 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0a9dd6-7e67-47f6-b3a3-98b50b06182d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:53.215466 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 150/298] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:53.221789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927d24c-4e69-42d7-a313-dae657632d35 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:53.272318 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4d789-bb9e-49f1-8ec2-853b91a82ba7 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:53.272634 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4d789-bb9e-49f1-8ec2-853b91a82ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:53.273469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4d789-bb9e-49f1-8ec2-853b91a82ba7 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:53.276346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 151/299] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:41:53.281562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5666b-158d-4ff7-90f2-e05cae0b1562 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:53.281844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5666b-158d-4ff7-90f2-e05cae0b1562 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:53.282118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5666b-158d-4ff7-90f2-e05cae0b1562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:53.301271 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-5bc5666b-158d-4ff7-90f2-e05cae0b1562 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:53.301603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:53.302466 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5bc5666b-158d-4ff7-90f2-e05cae0b1562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:53.347743 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927d24c-4e69-42d7-a313-dae657632d35 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:53.373944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 152/300] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:53.375200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71022f9c-f322-4188-b6fe-14fea6f17395 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:53.375484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71022f9c-f322-4188-b6fe-14fea6f17395 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:53.375755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71022f9c-f322-4188-b6fe-14fea6f17395 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:53.377657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 153/301] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:53.381168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f1560-57ac-4b32-b0ec-b6fb470547e8 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:53.381168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f1560-57ac-4b32-b0ec-b6fb470547e8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:53.381168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f1560-57ac-4b32-b0ec-b6fb470547e8 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:53.381168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 154/302] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:53.383034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff243a4-c454-4410-a5f0-cbce7f4f6120 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:53.383515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff243a4-c454-4410-a5f0-cbce7f4f6120 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:53.383975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff243a4-c454-4410-a5f0-cbce7f4f6120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:53.395507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3888e0-1cf1-413c-ac4b-3adb8358e28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:53.396157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3888e0-1cf1-413c-ac4b-3adb8358e28c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:53.396436 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3888e0-1cf1-413c-ac4b-3adb8358e28c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:53.396706 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3888e0-1cf1-413c-ac4b-3adb8358e28c None admin] PATH_INFO: `/v3/users/621389d663684029978592fbf244d1d4` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:53.397830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3888e0-1cf1-413c-ac4b-3adb8358e28c None admin] RBAC: Authorizing `identity:get_user(user_id=621389d663684029978592fbf244d1d4)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:53.399257 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3888e0-1cf1-413c-ac4b-3adb8358e28c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:53.400855 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 149/303] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:53 2022] GET /identity/v3/users/621389d663684029978592fbf244d1d4 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:53.402774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25952588-25bd-4bad-9cde-31ced0fbb700 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:53.403061 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25952588-25bd-4bad-9cde-31ced0fbb700 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:53.403335 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25952588-25bd-4bad-9cde-31ced0fbb700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:53.406036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7ff243a4-c454-4410-a5f0-cbce7f4f6120 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:53.407444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7ff243a4-c454-4410-a5f0-cbce7f4f6120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:53.418103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:53.424241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-25952588-25bd-4bad-9cde-31ced0fbb700 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:53.425462 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-25952588-25bd-4bad-9cde-31ced0fbb700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:53.482978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 155/304] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:53.493000 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff243a4-c454-4410-a5f0-cbce7f4f6120 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:53.495376 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 150/305] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:53.599183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff243a4-c454-4410-a5f0-cbce7f4f6120 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:53.634613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d9998-0f04-420a-8841-4cffb2d6d0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:53.635455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d9998-0f04-420a-8841-4cffb2d6d0f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:53.635869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d9998-0f04-420a-8841-4cffb2d6d0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:53.636324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d9998-0f04-420a-8841-4cffb2d6d0f9 None admin] PATH_INFO: `/v3/system/users/621389d663684029978592fbf244d1d4/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:53.637843 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d9998-0f04-420a-8841-4cffb2d6d0f9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=621389d663684029978592fbf244d1d4, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:53.639348 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d9998-0f04-420a-8841-4cffb2d6d0f9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:53.645155 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25952588-25bd-4bad-9cde-31ced0fbb700 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:53.656207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 156/306] 10.222.0.26 () {58 vars in 1334 bytes} [Wed Sep 7 07:41:53 2022] PUT /identity/v3/system/users/621389d663684029978592fbf244d1d4/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:41:53.722556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:53.758466 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d9998-0f04-420a-8841-4cffb2d6d0f9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:53.768793 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:53.769855 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25952588-25bd-4bad-9cde-31ced0fbb700 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:53.821517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd8612-43f8-4ad0-ae26-3089a3295db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:53.823351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd8612-43f8-4ad0-ae26-3089a3295db0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:53.823351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd8612-43f8-4ad0-ae26-3089a3295db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:53.823351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd8612-43f8-4ad0-ae26-3089a3295db0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:53.836342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd8612-43f8-4ad0-ae26-3089a3295db0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:53.841446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-e6dd8612-43f8-4ad0-ae26-3089a3295db0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:41:53.841446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:53.841446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:53.841446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:53.841446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:53.841446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:53.841446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:53.841446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:53.841446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:53.841446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:53.841446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:53.841446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:53.841446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:53.841446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:53.841446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:53.841446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:53.842319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:53.842319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:53.842319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:53.842319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:53.842319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:53.842319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:53.842319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:53.842319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:53.842319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:53.842319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:53.842319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:53.842319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:53.842319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:53.842319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:53.842319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:53.842319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:53.843080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:53.843080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:53.843080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:53.843080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:53.843080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:53.843080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:53.843080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:53.843080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:53.843080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:53.843080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:53.843080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:41:53.843080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:53.843080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 151/307] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:53.843080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd8612-43f8-4ad0-ae26-3089a3295db0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:53.887794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d9998-0f04-420a-8841-4cffb2d6d0f9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:53.926817 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72a4b2-2cfa-4588-94ce-40c323233168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:53.927456 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72a4b2-2cfa-4588-94ce-40c323233168 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:53.927772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72a4b2-2cfa-4588-94ce-40c323233168 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:53.927998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72a4b2-2cfa-4588-94ce-40c323233168 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:53.929500 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72a4b2-2cfa-4588-94ce-40c323233168 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:53.931244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72a4b2-2cfa-4588-94ce-40c323233168 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:53.936476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 157/308] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:53.936476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72a4b2-2cfa-4588-94ce-40c323233168 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:53.957536 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd8612-43f8-4ad0-ae26-3089a3295db0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:53.980823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:53.995483 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416bda8f-87fc-4c83-a3e9-8dd6278e3889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:53.996528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bda8f-87fc-4c83-a3e9-8dd6278e3889 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:53.996816 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bda8f-87fc-4c83-a3e9-8dd6278e3889 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:53.997234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bda8f-87fc-4c83-a3e9-8dd6278e3889 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:54.008012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416bda8f-87fc-4c83-a3e9-8dd6278e3889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:54.013092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-416bda8f-87fc-4c83-a3e9-8dd6278e3889 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:41:54.013092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:54.013092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:54.013092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:54.013092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:54.013092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:54.013092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:54.013092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:54.013092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:54.013092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:54.013092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:54.013092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:54.013092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:54.013092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:54.013092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:54.013092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:54.013955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:54.013955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:54.013955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:54.013955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:54.013955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:54.013955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:54.013955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:54.013955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:54.013955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:54.013955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:54.013955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:54.013955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:54.013955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:54.013955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:54.013955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:54.013955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:54.014711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:54.014711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:54.014711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:54.014711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:54.014711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:54.014711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:54.014711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:54.014711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:54.014711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:54.014711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:54.014711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:41:54.014711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:54.014711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 152/309] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:54.015356 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416bda8f-87fc-4c83-a3e9-8dd6278e3889 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:54.022035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:54.050094 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72a4b2-2cfa-4588-94ce-40c323233168 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:54.083899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11e04e-b08f-4752-9ce2-f61af020b485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:54.084532 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11e04e-b08f-4752-9ce2-f61af020b485 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:54.084807 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11e04e-b08f-4752-9ce2-f61af020b485 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:54.085080 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11e04e-b08f-4752-9ce2-f61af020b485 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:54.085826 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11e04e-b08f-4752-9ce2-f61af020b485 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:54.086126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11e04e-b08f-4752-9ce2-f61af020b485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:54.087611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11e04e-b08f-4752-9ce2-f61af020b485 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:54.096865 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 158/310] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:54.097683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11e04e-b08f-4752-9ce2-f61af020b485 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:54.127297 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416bda8f-87fc-4c83-a3e9-8dd6278e3889 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:54.164525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026990c7-8870-4fee-ac92-8cce1faa311a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:54.165123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026990c7-8870-4fee-ac92-8cce1faa311a None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:54.165401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026990c7-8870-4fee-ac92-8cce1faa311a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:54.165749 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026990c7-8870-4fee-ac92-8cce1faa311a None admin] PATH_INFO: `/v3/groups` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:54.166287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026990c7-8870-4fee-ac92-8cce1faa311a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:54.167734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026990c7-8870-4fee-ac92-8cce1faa311a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:54.183577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 153/311] 10.222.0.26 () {60 vars in 1134 bytes} [Wed Sep 7 07:41:54 2022] POST /identity/v3/groups => generated 229 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:41:54.188818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026990c7-8870-4fee-ac92-8cce1faa311a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:54.225491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11e04e-b08f-4752-9ce2-f61af020b485 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:54.251625 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:54.272725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8472b83-522c-4074-80f7-cdd03e99b953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:54.273707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8472b83-522c-4074-80f7-cdd03e99b953 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:54.274181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8472b83-522c-4074-80f7-cdd03e99b953 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:54.274660 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8472b83-522c-4074-80f7-cdd03e99b953 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:54.275632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8472b83-522c-4074-80f7-cdd03e99b953 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:54.276240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8472b83-522c-4074-80f7-cdd03e99b953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:54.277965 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8472b83-522c-4074-80f7-cdd03e99b953 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:54.281636 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:54.299952 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 159/312] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:54.304858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8472b83-522c-4074-80f7-cdd03e99b953 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:54.334816 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026990c7-8870-4fee-ac92-8cce1faa311a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:54.375049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6b3b7-c3f0-4639-a268-99daec3e2e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:54.375737 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6b3b7-c3f0-4639-a268-99daec3e2e58 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:54.376020 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6b3b7-c3f0-4639-a268-99daec3e2e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:54.376337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6b3b7-c3f0-4639-a268-99daec3e2e58 None admin] PATH_INFO: `/v3/users/f33af5039832446897e3f8f5d0350d7e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:54.396393 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6b3b7-c3f0-4639-a268-99daec3e2e58 None admin] RBAC: Authorizing `identity:get_user(user_id=f33af5039832446897e3f8f5d0350d7e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:54.398255 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6b3b7-c3f0-4639-a268-99daec3e2e58 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:54.400680 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 154/313] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:54 2022] GET /identity/v3/users/f33af5039832446897e3f8f5d0350d7e => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:54.407394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6b3b7-c3f0-4639-a268-99daec3e2e58 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:54.445045 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8472b83-522c-4074-80f7-cdd03e99b953 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:54.482268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40dbeb-bf16-4341-bcbe-f3164ca169d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:54.482901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40dbeb-bf16-4341-bcbe-f3164ca169d6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:54.483181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40dbeb-bf16-4341-bcbe-f3164ca169d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:54.483709 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40dbeb-bf16-4341-bcbe-f3164ca169d6 None admin] PATH_INFO: `/v3/users/63318198be344fbba10bc53d6c0c1bb9` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:54.484587 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40dbeb-bf16-4341-bcbe-f3164ca169d6 None admin] RBAC: Authorizing `identity:get_user(user_id=63318198be344fbba10bc53d6c0c1bb9)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:54.486911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40dbeb-bf16-4341-bcbe-f3164ca169d6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:54.488953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 160/314] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:54 2022] GET /identity/v3/users/63318198be344fbba10bc53d6c0c1bb9 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:54.494377 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40dbeb-bf16-4341-bcbe-f3164ca169d6 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:54.534296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:54.540687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6b3b7-c3f0-4639-a268-99daec3e2e58 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:54.571335 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:54.594666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fefce-9d43-4c3d-9228-802f1e20548e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:54.595844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fefce-9d43-4c3d-9228-802f1e20548e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:54.596675 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fefce-9d43-4c3d-9228-802f1e20548e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:54.597105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fefce-9d43-4c3d-9228-802f1e20548e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:54.597979 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fefce-9d43-4c3d-9228-802f1e20548e None admin] RBAC: Adding query filter params (role.id=a5350d487c374ea680a13ac3044168fc, user.id=f33af5039832446897e3f8f5d0350d7e, scope.system=all) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:54.598433 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fefce-9d43-4c3d-9228-802f1e20548e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:54.600760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fefce-9d43-4c3d-9228-802f1e20548e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:54.608191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 155/315] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Sep 7 07:41:54 2022] GET /identity/v3/role_assignments?user.id=f33af5039832446897e3f8f5d0350d7e&scope.system=all&role.id=a5350d487c374ea680a13ac3044168fc => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:54.616138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40dbeb-bf16-4341-bcbe-f3164ca169d6 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:54.657195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed1c9a-b5e6-4f21-b3c6-f705af60585a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:54.658284 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed1c9a-b5e6-4f21-b3c6-f705af60585a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:54.658805 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed1c9a-b5e6-4f21-b3c6-f705af60585a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:54.659221 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed1c9a-b5e6-4f21-b3c6-f705af60585a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:54.739358 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed1c9a-b5e6-4f21-b3c6-f705af60585a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:54.741138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed1c9a-b5e6-4f21-b3c6-f705af60585a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:54.812102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:54.851820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 161/316] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:41:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 07:41:54.857930 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fefce-9d43-4c3d-9228-802f1e20548e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:54.986350 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fefce-9d43-4c3d-9228-802f1e20548e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:55.027464 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974309c-cf9d-4f02-b030-8c2d6b75ad6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:55.028120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974309c-cf9d-4f02-b030-8c2d6b75ad6e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:55.028399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974309c-cf9d-4f02-b030-8c2d6b75ad6e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:55.028666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974309c-cf9d-4f02-b030-8c2d6b75ad6e None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:55.029794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974309c-cf9d-4f02-b030-8c2d6b75ad6e None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:55.031240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974309c-cf9d-4f02-b030-8c2d6b75ad6e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:55.033951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 156/317] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:41:54 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:55.039499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed1c9a-b5e6-4f21-b3c6-f705af60585a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:55.146401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed1c9a-b5e6-4f21-b3c6-f705af60585a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:55.182958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c58eda-9aba-44b1-aa2d-0f9a86263d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:55.183912 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c58eda-9aba-44b1-aa2d-0f9a86263d58 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:55.184413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c58eda-9aba-44b1-aa2d-0f9a86263d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:55.184904 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c58eda-9aba-44b1-aa2d-0f9a86263d58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:55.185852 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c58eda-9aba-44b1-aa2d-0f9a86263d58 None admin] RBAC: Adding query filter params (role.id=a5350d487c374ea680a13ac3044168fc, scope.project.id=ea579b172f214888a95d9af5a4e1d122, user.id=63318198be344fbba10bc53d6c0c1bb9) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:55.186365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c58eda-9aba-44b1-aa2d-0f9a86263d58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:55.188117 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c58eda-9aba-44b1-aa2d-0f9a86263d58 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:55.196828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 162/318] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:41:55 2022] GET /identity/v3/role_assignments?user.id=63318198be344fbba10bc53d6c0c1bb9&scope.project.id=ea579b172f214888a95d9af5a4e1d122&role.id=a5350d487c374ea680a13ac3044168fc => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:55.922790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76239bcc-4103-4b5c-9dd9-b3642dcdbb8a None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:55.923337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76239bcc-4103-4b5c-9dd9-b3642dcdbb8a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:55.923337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76239bcc-4103-4b5c-9dd9-b3642dcdbb8a None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:55.925430 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 157/319] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:55.931101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4261c07-d265-48a3-b913-825e3ebc4c22 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:55.931530 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4261c07-d265-48a3-b913-825e3ebc4c22 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:55.931719 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4261c07-d265-48a3-b913-825e3ebc4c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:55.949905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f4261c07-d265-48a3-b913-825e3ebc4c22 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:55.951126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f4261c07-d265-48a3-b913-825e3ebc4c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:55.976182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:56.020747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 163/320] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:56.288555 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76239bcc-4103-4b5c-9dd9-b3642dcdbb8a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:56.318197 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:56.397758 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76239bcc-4103-4b5c-9dd9-b3642dcdbb8a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:56.440764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c1fb9-36d7-45ab-a326-a8f4584a70e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:56.441393 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c1fb9-36d7-45ab-a326-a8f4584a70e4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:56.441671 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c1fb9-36d7-45ab-a326-a8f4584a70e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:56.441943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c1fb9-36d7-45ab-a326-a8f4584a70e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:56.452468 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c1fb9-36d7-45ab-a326-a8f4584a70e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:56.457818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-001c1fb9-36d7-45ab-a326-a8f4584a70e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:56.457818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:56.457818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:56.457818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:56.457818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:56.457818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:56.457818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:56.457818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:56.457818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:56.457818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:56.457818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:56.457818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:56.457818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:56.457818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:56.457818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:56.457818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:56.458650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:56.458650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:56.458650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:56.458650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:56.458650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:56.458650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:56.458650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:56.458650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:56.458650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:56.458650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:56.458650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:56.458650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:56.458650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:56.458650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:56.458650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:56.458650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:56.460896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:56.460896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:56.460896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:56.460896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:56.460896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:56.460896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:56.460896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:56.460896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:56.460896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:56.460896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:56.460896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:41:56.460896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:41:56.460896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 158/321] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:56.465073 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4261c07-d265-48a3-b913-825e3ebc4c22 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:56.520967 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c36477-1429-4cd9-a7e2-a263c933dfe6 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:56.521253 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c36477-1429-4cd9-a7e2-a263c933dfe6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:56.521526 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c36477-1429-4cd9-a7e2-a263c933dfe6 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:56.522996 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 159/322] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:56.527735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1bff8-7e68-462b-822c-6eb64344c480 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:56.528019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1bff8-7e68-462b-822c-6eb64344c480 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:56.528332 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1bff8-7e68-462b-822c-6eb64344c480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:56.547013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-92e1bff8-7e68-462b-822c-6eb64344c480 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:56.548194 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-92e1bff8-7e68-462b-822c-6eb64344c480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:56.575937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4261c07-d265-48a3-b913-825e3ebc4c22 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:56.613172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903956da-b2ab-4ba2-8417-3e5f8ec4ce46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:56.613779 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903956da-b2ab-4ba2-8417-3e5f8ec4ce46 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:56.614059 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903956da-b2ab-4ba2-8417-3e5f8ec4ce46 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:56.614328 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903956da-b2ab-4ba2-8417-3e5f8ec4ce46 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:56.614975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903956da-b2ab-4ba2-8417-3e5f8ec4ce46 None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:56.615280 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903956da-b2ab-4ba2-8417-3e5f8ec4ce46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:56.616683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903956da-b2ab-4ba2-8417-3e5f8ec4ce46 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:56.623942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 160/323] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:41:56.625183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 164/324] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:56.633267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1bff8-7e68-462b-822c-6eb64344c480 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:56.655749 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:56.734592 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1bff8-7e68-462b-822c-6eb64344c480 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:56.768665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfb2b1-4cc3-4b74-a3bc-e8785f0d669f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:56.769293 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfb2b1-4cc3-4b74-a3bc-e8785f0d669f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:56.769571 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfb2b1-4cc3-4b74-a3bc-e8785f0d669f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:56.769846 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfb2b1-4cc3-4b74-a3bc-e8785f0d669f None admin] PATH_INFO: `/v3/users/621389d663684029978592fbf244d1d4` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:56.770958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfb2b1-4cc3-4b74-a3bc-e8785f0d669f None admin] RBAC: Authorizing `identity:get_user(user_id=621389d663684029978592fbf244d1d4)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:56.772413 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfb2b1-4cc3-4b74-a3bc-e8785f0d669f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:56.774180 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 161/325] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:56 2022] GET /identity/v3/users/621389d663684029978592fbf244d1d4 => generated 293 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:56.779629 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903956da-b2ab-4ba2-8417-3e5f8ec4ce46 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:56.805332 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:56.888346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903956da-b2ab-4ba2-8417-3e5f8ec4ce46 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:56.898601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfb2b1-4cc3-4b74-a3bc-e8785f0d669f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:56.923016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6fdd2-5914-48dc-afcc-c38805514ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:56.924184 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6fdd2-5914-48dc-afcc-c38805514ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:56.924474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6fdd2-5914-48dc-afcc-c38805514ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:56.924745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6fdd2-5914-48dc-afcc-c38805514ae8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:56.925467 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f6fdd2-5914-48dc-afcc-c38805514ae8 None admin] RBAC: Adding query filter params (role.id=6ebbed85905048a198e4261a3510dafc, user.id=621389d663684029978592fbf244d1d4, scope.system=all) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:56.925769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f6fdd2-5914-48dc-afcc-c38805514ae8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:56.927236 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f6fdd2-5914-48dc-afcc-c38805514ae8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:56.939270 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 165/326] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Sep 7 07:41:56 2022] GET /identity/v3/role_assignments?user.id=621389d663684029978592fbf244d1d4&scope.system=all&role.id=6ebbed85905048a198e4261a3510dafc => generated 510 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:56.950748 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a4b00-54df-40e3-ae05-f4db5291787f None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:56.951035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a4b00-54df-40e3-ae05-f4db5291787f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:56.951310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a4b00-54df-40e3-ae05-f4db5291787f None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:56.952985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 166/327] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:41:56.959697 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0df77f-ad81-4115-b841-9209016e93fe None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:56.959990 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0df77f-ad81-4115-b841-9209016e93fe None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:56.960275 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0df77f-ad81-4115-b841-9209016e93fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:56.985447 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ec0df77f-ad81-4115-b841-9209016e93fe None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:41:56.987330 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ec0df77f-ad81-4115-b841-9209016e93fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:41:57.005025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:57.019243 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfb2b1-4cc3-4b74-a3bc-e8785f0d669f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:57.057948 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f951cea-2527-42b2-9c8e-c4670a38fda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:57.058852 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f951cea-2527-42b2-9c8e-c4670a38fda1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:57.059274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f951cea-2527-42b2-9c8e-c4670a38fda1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:57.059819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f951cea-2527-42b2-9c8e-c4670a38fda1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:57.061328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f951cea-2527-42b2-9c8e-c4670a38fda1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:57.062934 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f951cea-2527-42b2-9c8e-c4670a38fda1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:57.065105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 162/328] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:41:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:41:57.071054 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f951cea-2527-42b2-9c8e-c4670a38fda1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:57.080479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 167/329] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:41:57.177304 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f951cea-2527-42b2-9c8e-c4670a38fda1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:57.210395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857c4ac-11eb-4205-b472-9bcac68d3534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:57.210993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857c4ac-11eb-4205-b472-9bcac68d3534 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:57.211268 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857c4ac-11eb-4205-b472-9bcac68d3534 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:57.211535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857c4ac-11eb-4205-b472-9bcac68d3534 None admin] PATH_INFO: `/v3/groups` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:57.212180 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8857c4ac-11eb-4205-b472-9bcac68d3534 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:57.213637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8857c4ac-11eb-4205-b472-9bcac68d3534 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:57.233012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 163/330] 10.222.0.26 () {60 vars in 1134 bytes} [Wed Sep 7 07:41:57 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:41:57.358374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0df77f-ad81-4115-b841-9209016e93fe None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:57.371087 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:57.463219 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0df77f-ad81-4115-b841-9209016e93fe None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:57.499902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731e690-ae13-4ba2-8b2a-c196ebed37e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:57.500570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731e690-ae13-4ba2-8b2a-c196ebed37e6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:57.500851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731e690-ae13-4ba2-8b2a-c196ebed37e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:57.501129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731e690-ae13-4ba2-8b2a-c196ebed37e6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:57.512294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731e690-ae13-4ba2-8b2a-c196ebed37e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:41:57.517143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-8731e690-ae13-4ba2-8b2a-c196ebed37e6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:41:57.517143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:41:57.517143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:41:57.517143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:41:57.517143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:41:57.517143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:41:57.517143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:41:57.517143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:41:57.517143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:41:57.517143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:41:57.517143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:41:57.517143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:41:57.517143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:41:57.517143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:41:57.517143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:41:57.517143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:41:57.518441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:41:57.518441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:41:57.518441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:41:57.518441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:41:57.518441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:41:57.518441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:41:57.518441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:41:57.518441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:41:57.518441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:41:57.518441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:41:57.518441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:41:57.518441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:41:57.518441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:41:57.518441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:41:57.518441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:41:57.518441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:41:57.519223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:41:57.519223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:41:57.519223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:41:57.519223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:41:57.519223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:41:57.519223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:41:57.519223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:41:57.519223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:41:57.519223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:41:57.519223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:41:57.519223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:41:57.519223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:41:57.519223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 168/331] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:41:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:41:57.525620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857c4ac-11eb-4205-b472-9bcac68d3534 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:57.547828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:57.635292 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857c4ac-11eb-4205-b472-9bcac68d3534 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:57.672012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbd417-44d9-4fd1-a7da-b0576d545ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:57.672649 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbd417-44d9-4fd1-a7da-b0576d545ecd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:57.672923 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbd417-44d9-4fd1-a7da-b0576d545ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:57.673192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbd417-44d9-4fd1-a7da-b0576d545ecd None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:57.673967 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbd417-44d9-4fd1-a7da-b0576d545ecd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:41:57.674257 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbd417-44d9-4fd1-a7da-b0576d545ecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:57.675622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbd417-44d9-4fd1-a7da-b0576d545ecd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:57.686159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 164/332] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:41:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:57.692038 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731e690-ae13-4ba2-8b2a-c196ebed37e6 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:57.769077 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:57.800871 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731e690-ae13-4ba2-8b2a-c196ebed37e6 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:57.837505 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725e6d1-e26f-4c48-9d35-15d9bbfdd4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:57.837505 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725e6d1-e26f-4c48-9d35-15d9bbfdd4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:57.837766 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725e6d1-e26f-4c48-9d35-15d9bbfdd4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:57.838004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725e6d1-e26f-4c48-9d35-15d9bbfdd4a5 None admin] PATH_INFO: `/v3/users/f33af5039832446897e3f8f5d0350d7e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:57.839037 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725e6d1-e26f-4c48-9d35-15d9bbfdd4a5 None admin] RBAC: Authorizing `identity:get_user(user_id=f33af5039832446897e3f8f5d0350d7e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:57.840559 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725e6d1-e26f-4c48-9d35-15d9bbfdd4a5 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:57.842410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 169/333] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:41:57 2022] GET /identity/v3/users/f33af5039832446897e3f8f5d0350d7e => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:41:57.848439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbd417-44d9-4fd1-a7da-b0576d545ecd None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:41:57.948036 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:41:57.972755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbd417-44d9-4fd1-a7da-b0576d545ecd None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:41:58.007853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68645610-990c-4dbb-b0fb-60b5b3dc0952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:41:58.008524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68645610-990c-4dbb-b0fb-60b5b3dc0952 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:41:58.008795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68645610-990c-4dbb-b0fb-60b5b3dc0952 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:41:58.009063 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68645610-990c-4dbb-b0fb-60b5b3dc0952 None admin] PATH_INFO: `/v3/system/users/f33af5039832446897e3f8f5d0350d7e/roles/a5350d487c374ea680a13ac3044168fc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:41:58.010549 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68645610-990c-4dbb-b0fb-60b5b3dc0952 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f33af5039832446897e3f8f5d0350d7e, role_id=a5350d487c374ea680a13ac3044168fc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:41:58.011863 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68645610-990c-4dbb-b0fb-60b5b3dc0952 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:41:58.023937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 165/334] 10.222.0.26 () {58 vars in 1334 bytes} [Wed Sep 7 07:41:57 2022] PUT /identity/v3/system/users/f33af5039832446897e3f8f5d0350d7e/roles/a5350d487c374ea680a13ac3044168fc => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:42:00.019770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90330e2-fdf9-4aca-9552-d19109bfd439 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:00.020326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90330e2-fdf9-4aca-9552-d19109bfd439 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:00.020326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90330e2-fdf9-4aca-9552-d19109bfd439 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:00.021927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 170/335] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:00.028227 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5118f-6a86-4df2-ba05-0bf911a70739 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:00.029046 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5118f-6a86-4df2-ba05-0bf911a70739 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:00.029537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5118f-6a86-4df2-ba05-0bf911a70739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:00.057154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e3c5118f-6a86-4df2-ba05-0bf911a70739 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:00.058366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e3c5118f-6a86-4df2-ba05-0bf911a70739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:00.139526 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 166/336] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:42:00.168932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f21e85-4b63-469f-bf21-20362362517c None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:00.169572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f21e85-4b63-469f-bf21-20362362517c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:00.170043 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f21e85-4b63-469f-bf21-20362362517c None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:00.171826 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 171/337] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:00.177704 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dedd4e-d508-48f4-b0ba-15ea4b58b528 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:00.177992 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dedd4e-d508-48f4-b0ba-15ea4b58b528 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:00.178270 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dedd4e-d508-48f4-b0ba-15ea4b58b528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:00.204331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b2dedd4e-d508-48f4-b0ba-15ea4b58b528 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:00.205555 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b2dedd4e-d508-48f4-b0ba-15ea4b58b528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:00.298465 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 167/338] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:00.522462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca587b-b37a-4b11-86e5-adb769a27c26 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:00.523161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca587b-b37a-4b11-86e5-adb769a27c26 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:00.523644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca587b-b37a-4b11-86e5-adb769a27c26 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:00.525446 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 172/339] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:00.531298 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15430aad-7c45-482a-b3ce-e462baaebb0c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:00.531582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15430aad-7c45-482a-b3ce-e462baaebb0c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:00.531859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15430aad-7c45-482a-b3ce-e462baaebb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:00.543924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:00.559411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-15430aad-7c45-482a-b3ce-e462baaebb0c None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:00.561081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-15430aad-7c45-482a-b3ce-e462baaebb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:00.650951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 168/340] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:00.719611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca587b-b37a-4b11-86e5-adb769a27c26 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:00.851762 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15430aad-7c45-482a-b3ce-e462baaebb0c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:00.862367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca587b-b37a-4b11-86e5-adb769a27c26 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:00.916731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449fca4-733b-4b95-bb3c-3c8a1010b4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:00.917477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449fca4-733b-4b95-bb3c-3c8a1010b4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:00.917752 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449fca4-733b-4b95-bb3c-3c8a1010b4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:00.918028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449fca4-733b-4b95-bb3c-3c8a1010b4b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:00.930444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:00.942390 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449fca4-733b-4b95-bb3c-3c8a1010b4b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:00.948165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-4449fca4-733b-4b95-bb3c-3c8a1010b4b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:42:00.948165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:00.948165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:00.948165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:00.948165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:00.948165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:00.948165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:00.948165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:00.948165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:00.948165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:00.948165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:00.948165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:00.948165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:00.948165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:00.948165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:00.948165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:00.949040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:00.949040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:00.949040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:00.949040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:00.949040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:00.949040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:00.949040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:00.949040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:00.949040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:00.949040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:00.949040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:00.949040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:00.949040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:00.949040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:00.949040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:00.949040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:00.949811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:00.949811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:00.949811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:00.949811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:00.949811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:00.949811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:00.949811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:00.949811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:00.949811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:00.949811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:00.949811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:42:00.949811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:00.949811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 173/341] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:00.958056 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449fca4-733b-4b95-bb3c-3c8a1010b4b6 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:01.050714 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15430aad-7c45-482a-b3ce-e462baaebb0c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:01.121174 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449fca4-733b-4b95-bb3c-3c8a1010b4b6 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:01.135278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:01.142636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe71241-8cae-4774-b0c8-77f6dc0b994c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:01.143287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe71241-8cae-4774-b0c8-77f6dc0b994c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:01.143566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe71241-8cae-4774-b0c8-77f6dc0b994c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:01.143839 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe71241-8cae-4774-b0c8-77f6dc0b994c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:01.169118 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe71241-8cae-4774-b0c8-77f6dc0b994c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:01.174797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-2fe71241-8cae-4774-b0c8-77f6dc0b994c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:42:01.174797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:01.174797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:01.174797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:01.174797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:01.174797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:01.174797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:01.174797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:01.174797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:01.174797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:01.174797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:01.174797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:01.174797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:01.174797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:01.174797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:01.174797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:01.175808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:01.175808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:01.175808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:01.175808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:01.175808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:01.175808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:01.175808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:01.175808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:01.175808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:01.175808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:01.175808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:01.175808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:01.175808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:01.175808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:01.175808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:01.175808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:01.176626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:01.176626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:01.176626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:01.176626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:01.176626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:01.176626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:01.176626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:01.176626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:01.176626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:01.176626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:01.176626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:42:01.176626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:01.176626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 169/342] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:42:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:42:01.190113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe71241-8cae-4774-b0c8-77f6dc0b994c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:01.192438 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8597e-11ec-42a1-9d9d-ffd31cedcf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:01.195745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8597e-11ec-42a1-9d9d-ffd31cedcf67 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:01.196025 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8597e-11ec-42a1-9d9d-ffd31cedcf67 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:01.197146 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8597e-11ec-42a1-9d9d-ffd31cedcf67 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:01.197877 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8597e-11ec-42a1-9d9d-ffd31cedcf67 None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:01.198186 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8597e-11ec-42a1-9d9d-ffd31cedcf67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:01.199591 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8597e-11ec-42a1-9d9d-ffd31cedcf67 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:01.217001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 174/343] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:42:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:01.218138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8597e-11ec-42a1-9d9d-ffd31cedcf67 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:01.312459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:01.364864 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8597e-11ec-42a1-9d9d-ffd31cedcf67 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:01.428367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b284b-c612-43c7-8082-661a0c83597f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:01.429173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b284b-c612-43c7-8082-661a0c83597f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:01.429453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b284b-c612-43c7-8082-661a0c83597f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:01.429726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b284b-c612-43c7-8082-661a0c83597f None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:01.430396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b284b-c612-43c7-8082-661a0c83597f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:01.430694 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b284b-c612-43c7-8082-661a0c83597f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:01.432540 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b284b-c612-43c7-8082-661a0c83597f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:01.436834 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe71241-8cae-4774-b0c8-77f6dc0b994c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:01.444822 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 175/344] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:01.446770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b284b-c612-43c7-8082-661a0c83597f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:01.485221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d23a6f-865c-4d22-94f7-dc428ef9376e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:01.486596 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d23a6f-865c-4d22-94f7-dc428ef9376e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:01.486876 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d23a6f-865c-4d22-94f7-dc428ef9376e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:01.487147 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d23a6f-865c-4d22-94f7-dc428ef9376e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:01.500692 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d23a6f-865c-4d22-94f7-dc428ef9376e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:01.506138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-c7d23a6f-865c-4d22-94f7-dc428ef9376e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:42:01.506138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:01.506138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:01.506138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:01.506138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:01.506138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:01.506138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:01.506138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:01.506138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:01.506138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:01.506138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:01.506138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:01.506138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:01.506138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:01.506138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:01.506138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:01.507217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:01.507217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:01.507217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:01.507217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:01.507217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:01.507217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:01.507217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:01.507217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:01.507217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:01.507217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:01.507217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:01.507217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:01.507217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:01.507217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:01.507217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:01.507217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:01.508296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:01.508296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:01.508296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:01.508296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:01.508296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:01.508296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:01.508296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:01.508296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:01.508296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:01.508296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:01.508296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:42:01.508296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:01.510167 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 170/345] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:01.516689 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd132874-b626-4989-9337-497830f64f72 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:01.517119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd132874-b626-4989-9337-497830f64f72 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:01.518279 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd132874-b626-4989-9337-497830f64f72 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:01.519740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 171/346] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:01.520679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd132874-b626-4989-9337-497830f64f72 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:01.570298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b284b-c612-43c7-8082-661a0c83597f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:01.586608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:01.598611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:01.615127 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d14fb6-b138-4556-9d11-7becaf8ed0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:01.616180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d14fb6-b138-4556-9d11-7becaf8ed0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:01.616653 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d14fb6-b138-4556-9d11-7becaf8ed0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:01.617138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d14fb6-b138-4556-9d11-7becaf8ed0b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:01.652487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd132874-b626-4989-9337-497830f64f72 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:01.701750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f98c3-139c-4fc6-a692-804c0e1ece78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:01.702720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f98c3-139c-4fc6-a692-804c0e1ece78 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:01.703240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f98c3-139c-4fc6-a692-804c0e1ece78 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:01.703510 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f98c3-139c-4fc6-a692-804c0e1ece78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:01.710098 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d14fb6-b138-4556-9d11-7becaf8ed0b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:01.711608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d14fb6-b138-4556-9d11-7becaf8ed0b1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:01.784213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f98c3-139c-4fc6-a692-804c0e1ece78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:01.786215 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f98c3-139c-4fc6-a692-804c0e1ece78 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:01.841505 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 176/347] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:01.844538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d14fb6-b138-4556-9d11-7becaf8ed0b1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:01.857302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:01.873883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:01.915951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 172/348] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:01.916963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d137aa8-1c8f-40e2-afd1-ed2526badebe None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:01.917240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d137aa8-1c8f-40e2-afd1-ed2526badebe None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:01.917673 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d137aa8-1c8f-40e2-afd1-ed2526badebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:01.945253 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6d137aa8-1c8f-40e2-afd1-ed2526badebe None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:01.946566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6d137aa8-1c8f-40e2-afd1-ed2526badebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:01.974850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d14fb6-b138-4556-9d11-7becaf8ed0b1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:02.025229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fac7e5-5d48-499c-80dc-b33b439739ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:02.026102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fac7e5-5d48-499c-80dc-b33b439739ba None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:02.026458 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fac7e5-5d48-499c-80dc-b33b439739ba None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:02.026725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fac7e5-5d48-499c-80dc-b33b439739ba None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:02.027328 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fac7e5-5d48-499c-80dc-b33b439739ba None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:02.027629 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fac7e5-5d48-499c-80dc-b33b439739ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:02.029086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fac7e5-5d48-499c-80dc-b33b439739ba None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:02.032716 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 173/349] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:02.036626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ab72-3a6b-4b60-ae30-f8e9ef82800d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:02.036626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ab72-3a6b-4b60-ae30-f8e9ef82800d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:02.036626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ab72-3a6b-4b60-ae30-f8e9ef82800d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:02.036626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 174/350] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:02.038210 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519ab72-3a6b-4b60-ae30-f8e9ef82800d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:02.045336 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 177/351] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:42:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:02.047003 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fac7e5-5d48-499c-80dc-b33b439739ba None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:02.151060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:02.170806 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:02.179338 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519ab72-3a6b-4b60-ae30-f8e9ef82800d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:02.193820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fac7e5-5d48-499c-80dc-b33b439739ba None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:02.227918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5daa42-2e74-42b0-9bce-4b36b16c8e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:02.228583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5daa42-2e74-42b0-9bce-4b36b16c8e6e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:02.228859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5daa42-2e74-42b0-9bce-4b36b16c8e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:02.229133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5daa42-2e74-42b0-9bce-4b36b16c8e6e None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:02.230333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5daa42-2e74-42b0-9bce-4b36b16c8e6e None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:02.232102 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5daa42-2e74-42b0-9bce-4b36b16c8e6e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:02.233078 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae5a3a-5dfb-432b-b55f-1fbda78c40f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:02.233910 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae5a3a-5dfb-432b-b55f-1fbda78c40f2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:02.234018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 175/352] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:42:02 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:02.234401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae5a3a-5dfb-432b-b55f-1fbda78c40f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:02.234803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae5a3a-5dfb-432b-b55f-1fbda78c40f2 None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:02.235292 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed005d1-f0fc-4b24-9829-bec80b7e2480 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:02.235571 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed005d1-f0fc-4b24-9829-bec80b7e2480 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:02.236845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae5a3a-5dfb-432b-b55f-1fbda78c40f2 None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:02.237018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed005d1-f0fc-4b24-9829-bec80b7e2480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:02.238613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae5a3a-5dfb-432b-b55f-1fbda78c40f2 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:02.240785 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 178/353] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:42:02 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:02.241773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae5a3a-5dfb-432b-b55f-1fbda78c40f2 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:02.258571 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-fed005d1-f0fc-4b24-9829-bec80b7e2480 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:02.259769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fed005d1-f0fc-4b24-9829-bec80b7e2480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:02.351948 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 176/354] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:02.353561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503bebe-ac2f-4781-bd78-1e0df33805e6 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:02.354047 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503bebe-ac2f-4781-bd78-1e0df33805e6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:02.354493 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503bebe-ac2f-4781-bd78-1e0df33805e6 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:02.356165 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 177/355] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:02.357025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503bebe-ac2f-4781-bd78-1e0df33805e6 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:02.379853 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae5a3a-5dfb-432b-b55f-1fbda78c40f2 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:02.422522 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40e123-68bf-4dab-ae6b-c361a3988f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:02.422522 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40e123-68bf-4dab-ae6b-c361a3988f4e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:02.422759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40e123-68bf-4dab-ae6b-c361a3988f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:02.423095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40e123-68bf-4dab-ae6b-c361a3988f4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:02.450294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:02.463784 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:02.486949 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503bebe-ac2f-4781-bd78-1e0df33805e6 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:02.511825 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40e123-68bf-4dab-ae6b-c361a3988f4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:02.513357 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40e123-68bf-4dab-ae6b-c361a3988f4e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:02.533666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1769d-03ff-4e55-8fa7-c1e0bbe025db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:02.534306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1769d-03ff-4e55-8fa7-c1e0bbe025db None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:02.534582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1769d-03ff-4e55-8fa7-c1e0bbe025db None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:02.534849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1769d-03ff-4e55-8fa7-c1e0bbe025db None admin] PATH_INFO: `/v3/groups/a06ef62241eb4ae4b5ccbec174139142` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:02.542461 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1769d-03ff-4e55-8fa7-c1e0bbe025db None admin] RBAC: Authorizing `identity:get_group(group_id=a06ef62241eb4ae4b5ccbec174139142)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:02.545986 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1769d-03ff-4e55-8fa7-c1e0bbe025db None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:02.546698 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 178/356] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 7 07:42:02 2022] GET /identity/v3/groups/a06ef62241eb4ae4b5ccbec174139142 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:02.552141 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1769d-03ff-4e55-8fa7-c1e0bbe025db None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:02.642354 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 179/357] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:02.643510 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3d17e-38b8-431c-8502-d814e63b7230 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:02.643859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3d17e-38b8-431c-8502-d814e63b7230 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:02.644070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3d17e-38b8-431c-8502-d814e63b7230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:02.663598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c4b3d17e-38b8-431c-8502-d814e63b7230 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:02.665070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c4b3d17e-38b8-431c-8502-d814e63b7230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:02.679866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1769d-03ff-4e55-8fa7-c1e0bbe025db None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:02.720193 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03a216-f382-45d0-98fc-4c254ae51353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:02.721304 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03a216-f382-45d0-98fc-4c254ae51353 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:02.721587 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03a216-f382-45d0-98fc-4c254ae51353 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:02.721866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03a216-f382-45d0-98fc-4c254ae51353 None admin] PATH_INFO: `/v3/groups/a06ef62241eb4ae4b5ccbec174139142` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:02.722990 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03a216-f382-45d0-98fc-4c254ae51353 None admin] RBAC: Authorizing `identity:get_group(group_id=a06ef62241eb4ae4b5ccbec174139142)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:02.723401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:02.724534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03a216-f382-45d0-98fc-4c254ae51353 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:02.726308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 179/358] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 7 07:42:02 2022] GET /identity/v3/groups/a06ef62241eb4ae4b5ccbec174139142 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:02.728125 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03a216-f382-45d0-98fc-4c254ae51353 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:02.745396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:02.751607 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 180/359] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:02.752361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3d17e-38b8-431c-8502-d814e63b7230 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:02.872303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03a216-f382-45d0-98fc-4c254ae51353 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:02.887707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3d17e-38b8-431c-8502-d814e63b7230 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:02.922029 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91d0a0-b790-4f0e-b04f-5c6d33c40dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:02.922248 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91d0a0-b790-4f0e-b04f-5c6d33c40dee None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:02.922517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91d0a0-b790-4f0e-b04f-5c6d33c40dee None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:02.922785 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91d0a0-b790-4f0e-b04f-5c6d33c40dee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:02.934019 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b69e7-bf13-4eb5-9d64-3bebab7c682a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:02.934559 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b69e7-bf13-4eb5-9d64-3bebab7c682a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:02.934836 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b69e7-bf13-4eb5-9d64-3bebab7c682a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:02.935104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b69e7-bf13-4eb5-9d64-3bebab7c682a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:02.935875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b69e7-bf13-4eb5-9d64-3bebab7c682a None admin] RBAC: Adding query filter params (group.id=a06ef62241eb4ae4b5ccbec174139142, role.id=6ebbed85905048a198e4261a3510dafc, scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:02.936921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b69e7-bf13-4eb5-9d64-3bebab7c682a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:02.937029 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab91d0a0-b790-4f0e-b04f-5c6d33c40dee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:02.937588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b69e7-bf13-4eb5-9d64-3bebab7c682a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:02.942618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ab91d0a0-b790-4f0e-b04f-5c6d33c40dee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:42:02.942618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:02.942618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:02.942618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:02.942618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:02.942618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:02.942618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:02.942618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:02.942618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:02.942618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:02.942618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:02.942618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:02.942618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:02.942618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:02.942618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:02.942618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:02.943497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:02.943497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:02.943497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:02.943497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:02.943497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:02.943497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:02.943497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:02.943497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:02.943497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:02.943497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:02.943497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:02.943497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:02.943497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:02.943497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:02.943497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:02.943497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:02.944983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:02.944983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:02.944983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:02.944983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:02.944983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:02.944983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:02.944983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:02.944983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:02.944983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:02.944983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:02.944983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:42:02.944983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:02.944983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 180/360] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:42:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:02.945659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91d0a0-b790-4f0e-b04f-5c6d33c40dee None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:02.948230 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 181/361] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 7 07:42:02 2022] GET /identity/v3/role_assignments?group.id=a06ef62241eb4ae4b5ccbec174139142&scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5&role.id=6ebbed85905048a198e4261a3510dafc => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:02.948230 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b69e7-bf13-4eb5-9d64-3bebab7c682a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:03.021765 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:03.050741 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:03.094550 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91d0a0-b790-4f0e-b04f-5c6d33c40dee None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:03.105319 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b69e7-bf13-4eb5-9d64-3bebab7c682a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:03.141659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699c106-3b44-45ca-8a88-70f55949787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:03.142830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699c106-3b44-45ca-8a88-70f55949787f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:03.143091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699c106-3b44-45ca-8a88-70f55949787f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:03.143364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699c106-3b44-45ca-8a88-70f55949787f None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:03.145196 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699c106-3b44-45ca-8a88-70f55949787f None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:03.147308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699c106-3b44-45ca-8a88-70f55949787f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:03.148663 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 181/362] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:42:02 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:03.149497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699c106-3b44-45ca-8a88-70f55949787f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:03.152595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b564922-ac44-45dd-bacd-68161f470a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:03.153236 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b564922-ac44-45dd-bacd-68161f470a08 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:03.153515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b564922-ac44-45dd-bacd-68161f470a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:03.153778 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b564922-ac44-45dd-bacd-68161f470a08 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:03.168611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b564922-ac44-45dd-bacd-68161f470a08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:03.174129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-5b564922-ac44-45dd-bacd-68161f470a08 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:42:03.174129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:03.174129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:03.174129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:03.174129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:03.174129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:03.174129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:03.174129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:03.174129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:03.174129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:03.174129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:03.174129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:03.174129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:03.174129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:03.174129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:03.174129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:03.175395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:03.175395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:03.175395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:03.175395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:03.175395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:03.175395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:03.175395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:03.175395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:03.175395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:03.175395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:03.175395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:03.175395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:03.175395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:03.175395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:03.175395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:03.175395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:03.176229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:03.176229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:03.176229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:03.176229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:03.176229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:03.176229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:03.176229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:03.176229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:03.176229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:03.176229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:03.176229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 07:42:03.176229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:03.176229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 182/363] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:03.176905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b564922-ac44-45dd-bacd-68161f470a08 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:03.276961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699c106-3b44-45ca-8a88-70f55949787f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:03.302824 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b564922-ac44-45dd-bacd-68161f470a08 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:03.312557 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:03.323026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586baaf-744f-4e5a-ad96-c342ad462e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:03.323636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586baaf-744f-4e5a-ad96-c342ad462e68 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:03.323913 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586baaf-744f-4e5a-ad96-c342ad462e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:03.324210 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586baaf-744f-4e5a-ad96-c342ad462e68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:03.324948 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586baaf-744f-4e5a-ad96-c342ad462e68 None admin] RBAC: Adding query filter params (group.id=a06ef62241eb4ae4b5ccbec174139142, role.id=84a30faea3e94eb48b03ec7ceb7c1392, scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:03.325251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586baaf-744f-4e5a-ad96-c342ad462e68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:03.326691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586baaf-744f-4e5a-ad96-c342ad462e68 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:03.332644 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 182/364] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 7 07:42:03 2022] GET /identity/v3/role_assignments?group.id=a06ef62241eb4ae4b5ccbec174139142&scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5&role.id=84a30faea3e94eb48b03ec7ceb7c1392 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:03.333548 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586baaf-744f-4e5a-ad96-c342ad462e68 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:03.338276 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:03.359043 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3600f-4dd0-4670-b4d1-eb063d9d723c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:03.360030 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3600f-4dd0-4670-b4d1-eb063d9d723c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:03.360594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3600f-4dd0-4670-b4d1-eb063d9d723c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:03.362369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3600f-4dd0-4670-b4d1-eb063d9d723c None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:03.363534 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3600f-4dd0-4670-b4d1-eb063d9d723c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:03.364167 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3600f-4dd0-4670-b4d1-eb063d9d723c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:03.366605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3600f-4dd0-4670-b4d1-eb063d9d723c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:03.378357 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 183/365] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:03.379388 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3600f-4dd0-4670-b4d1-eb063d9d723c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:03.487828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586baaf-744f-4e5a-ad96-c342ad462e68 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:03.539664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e7324-fd72-4375-801f-3a40225bf4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:03.540284 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e7324-fd72-4375-801f-3a40225bf4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:03.540564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e7324-fd72-4375-801f-3a40225bf4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:03.540837 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e7324-fd72-4375-801f-3a40225bf4c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:03.543560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3600f-4dd0-4670-b4d1-eb063d9d723c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:03.557773 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e7324-fd72-4375-801f-3a40225bf4c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:03.563213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-456e7324-fd72-4375-801f-3a40225bf4c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:42:03.563213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:03.563213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:03.563213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:03.563213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:03.563213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:03.563213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:03.563213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:03.563213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:03.563213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:03.563213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:03.563213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:03.563213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:03.563213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:03.563213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:03.563213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:03.565130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:03.565130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:03.565130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:03.565130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:03.565130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:03.565130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:03.565130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:03.565130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:03.565130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:03.565130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:03.565130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:03.565130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:03.565130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:03.565130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:03.565130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:03.565130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:03.566011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:03.566011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:03.566011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:03.566011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:03.566011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:03.566011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:03.566011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:03.566011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:03.566011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:03.566011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:03.566011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:42:03.566011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:03.566011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 183/366] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:42:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:03.566011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e7324-fd72-4375-801f-3a40225bf4c9 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:03.597809 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b48f6b-9a73-4ae7-9f78-6d661b2f8f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:03.598767 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b48f6b-9a73-4ae7-9f78-6d661b2f8f1a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:03.599333 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b48f6b-9a73-4ae7-9f78-6d661b2f8f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:03.599799 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b48f6b-9a73-4ae7-9f78-6d661b2f8f1a None admin] PATH_INFO: `/v3/groups/a06ef62241eb4ae4b5ccbec174139142` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:03.601351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b48f6b-9a73-4ae7-9f78-6d661b2f8f1a None admin] RBAC: Authorizing `identity:get_group(group_id=a06ef62241eb4ae4b5ccbec174139142)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:03.604339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b48f6b-9a73-4ae7-9f78-6d661b2f8f1a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:03.606466 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 184/367] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 7 07:42:03 2022] GET /identity/v3/groups/a06ef62241eb4ae4b5ccbec174139142 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:03.607527 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b48f6b-9a73-4ae7-9f78-6d661b2f8f1a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:03.636846 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:03.655035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:03.678795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e7324-fd72-4375-801f-3a40225bf4c9 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:03.722179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46439b73-182f-4943-9243-e452bf2df05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:03.722810 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46439b73-182f-4943-9243-e452bf2df05c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:03.723096 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46439b73-182f-4943-9243-e452bf2df05c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:03.723371 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46439b73-182f-4943-9243-e452bf2df05c None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:03.724044 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46439b73-182f-4943-9243-e452bf2df05c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:03.724380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46439b73-182f-4943-9243-e452bf2df05c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:03.725747 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46439b73-182f-4943-9243-e452bf2df05c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:03.740171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 184/368] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:42:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:03.741357 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46439b73-182f-4943-9243-e452bf2df05c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:03.746169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b48f6b-9a73-4ae7-9f78-6d661b2f8f1a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:03.798963 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31e500-1b2e-4136-b34f-44a0b126ab83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:03.800577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31e500-1b2e-4136-b34f-44a0b126ab83 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:03.801075 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31e500-1b2e-4136-b34f-44a0b126ab83 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:03.801567 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31e500-1b2e-4136-b34f-44a0b126ab83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:03.877702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46439b73-182f-4943-9243-e452bf2df05c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:03.891954 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31e500-1b2e-4136-b34f-44a0b126ab83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:03.894559 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31e500-1b2e-4136-b34f-44a0b126ab83 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:03.916550 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:03.926114 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65ad21-1155-4778-b1cb-cdca5c928738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:03.926718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65ad21-1155-4778-b1cb-cdca5c928738 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:03.927020 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65ad21-1155-4778-b1cb-cdca5c928738 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:03.927293 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65ad21-1155-4778-b1cb-cdca5c928738 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:03.929811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65ad21-1155-4778-b1cb-cdca5c928738 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:03.930126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65ad21-1155-4778-b1cb-cdca5c928738 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:03.931518 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65ad21-1155-4778-b1cb-cdca5c928738 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:03.944866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 185/369] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:03.945500 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:03.945695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65ad21-1155-4778-b1cb-cdca5c928738 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:04.027520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 185/370] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:04.028406 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31e500-1b2e-4136-b34f-44a0b126ab83 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:04.073392 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65ad21-1155-4778-b1cb-cdca5c928738 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:04.110316 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdf55c-a5e4-4856-99d2-8d9cfa5a2e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:04.111087 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdf55c-a5e4-4856-99d2-8d9cfa5a2e09 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:04.111404 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdf55c-a5e4-4856-99d2-8d9cfa5a2e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:04.111636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdf55c-a5e4-4856-99d2-8d9cfa5a2e09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:04.112418 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bdf55c-a5e4-4856-99d2-8d9cfa5a2e09 None admin] RBAC: Adding query filter params (group.id=a06ef62241eb4ae4b5ccbec174139142, role.id=6ebbed85905048a198e4261a3510dafc, scope.project.id=ea579b172f214888a95d9af5a4e1d122) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:04.112827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bdf55c-a5e4-4856-99d2-8d9cfa5a2e09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:04.114357 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bdf55c-a5e4-4856-99d2-8d9cfa5a2e09 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:04.120479 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 186/371] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 7 07:42:03 2022] GET /identity/v3/role_assignments?group.id=a06ef62241eb4ae4b5ccbec174139142&scope.project.id=ea579b172f214888a95d9af5a4e1d122&role.id=6ebbed85905048a198e4261a3510dafc => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:04.121417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdf55c-a5e4-4856-99d2-8d9cfa5a2e09 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:04.159319 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31e500-1b2e-4136-b34f-44a0b126ab83 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:04.180918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:04.228889 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93677e5-6d90-4ad4-84a2-fe6e7d546d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:04.229555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93677e5-6d90-4ad4-84a2-fe6e7d546d9d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:04.229820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93677e5-6d90-4ad4-84a2-fe6e7d546d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:04.230101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93677e5-6d90-4ad4-84a2-fe6e7d546d9d None admin] PATH_INFO: `/v3/users/f33af5039832446897e3f8f5d0350d7e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:04.231391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93677e5-6d90-4ad4-84a2-fe6e7d546d9d None admin] RBAC: Authorizing `identity:get_user(user_id=f33af5039832446897e3f8f5d0350d7e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:04.233009 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93677e5-6d90-4ad4-84a2-fe6e7d546d9d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:04.234805 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 186/372] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:42:04 2022] GET /identity/v3/users/f33af5039832446897e3f8f5d0350d7e => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:04.235584 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93677e5-6d90-4ad4-84a2-fe6e7d546d9d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:04.240478 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:04.245045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdf55c-a5e4-4856-99d2-8d9cfa5a2e09 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:04.290765 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f83a4-0f7f-4b3b-90fd-059cdda82dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:04.292466 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f83a4-0f7f-4b3b-90fd-059cdda82dfc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:04.293715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f83a4-0f7f-4b3b-90fd-059cdda82dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:04.294179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f83a4-0f7f-4b3b-90fd-059cdda82dfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:04.445958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f83a4-0f7f-4b3b-90fd-059cdda82dfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:04.447486 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f83a4-0f7f-4b3b-90fd-059cdda82dfc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:04.450551 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93677e5-6d90-4ad4-84a2-fe6e7d546d9d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:04.490337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeaf9a1-6b0e-4797-ad04-bcb8881883a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:04.490976 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaf9a1-6b0e-4797-ad04-bcb8881883a2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:04.491252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaf9a1-6b0e-4797-ad04-bcb8881883a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:04.491522 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaf9a1-6b0e-4797-ad04-bcb8881883a2 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:04.492911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeaf9a1-6b0e-4797-ad04-bcb8881883a2 None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:04.494560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeaf9a1-6b0e-4797-ad04-bcb8881883a2 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:04.496645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 187/373] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:42:04 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:04.497489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeaf9a1-6b0e-4797-ad04-bcb8881883a2 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:04.507919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:04.576719 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:04.603870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 187/374] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:04.605900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f83a4-0f7f-4b3b-90fd-059cdda82dfc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:04.634909 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeaf9a1-6b0e-4797-ad04-bcb8881883a2 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:04.685552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328e0fe-ce6c-4608-b440-79c6261a0a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:04.686541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328e0fe-ce6c-4608-b440-79c6261a0a24 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:04.687024 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328e0fe-ce6c-4608-b440-79c6261a0a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:04.687504 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328e0fe-ce6c-4608-b440-79c6261a0a24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:04.688465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328e0fe-ce6c-4608-b440-79c6261a0a24 None admin] RBAC: Adding query filter params (role.id=a5350d487c374ea680a13ac3044168fc, user.id=f33af5039832446897e3f8f5d0350d7e, scope.system=all) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:04.689657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328e0fe-ce6c-4608-b440-79c6261a0a24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:04.691550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328e0fe-ce6c-4608-b440-79c6261a0a24 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:04.705843 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 188/375] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Sep 7 07:42:04 2022] GET /identity/v3/role_assignments?user.id=f33af5039832446897e3f8f5d0350d7e&scope.system=all&role.id=a5350d487c374ea680a13ac3044168fc => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:04.706993 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328e0fe-ce6c-4608-b440-79c6261a0a24 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:04.825121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328e0fe-ce6c-4608-b440-79c6261a0a24 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:04.839245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f83a4-0f7f-4b3b-90fd-059cdda82dfc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:04.859707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:04.868257 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4929959-6b01-4333-b9e2-2cca87f46df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:04.869285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4929959-6b01-4333-b9e2-2cca87f46df2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:04.869783 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4929959-6b01-4333-b9e2-2cca87f46df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:04.870251 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4929959-6b01-4333-b9e2-2cca87f46df2 None admin] PATH_INFO: `/v3/projects/bb18598c40b6447faef02991e1b9c4ed` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:04.871645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4929959-6b01-4333-b9e2-2cca87f46df2 None admin] RBAC: Authorizing `identity:get_project(project_id=bb18598c40b6447faef02991e1b9c4ed)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:04.873355 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4929959-6b01-4333-b9e2-2cca87f46df2 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:04.875563 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 189/376] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:42:04 2022] GET /identity/v3/projects/bb18598c40b6447faef02991e1b9c4ed => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:04.881917 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4929959-6b01-4333-b9e2-2cca87f46df2 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:04.927580 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daff634-b343-4702-b26b-70a05e5bf1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:04.928171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daff634-b343-4702-b26b-70a05e5bf1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:04.928456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daff634-b343-4702-b26b-70a05e5bf1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:04.928732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daff634-b343-4702-b26b-70a05e5bf1a4 None admin] PATH_INFO: `/v3/groups/a06ef62241eb4ae4b5ccbec174139142` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:04.936102 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daff634-b343-4702-b26b-70a05e5bf1a4 None admin] RBAC: Authorizing `identity:get_group(group_id=a06ef62241eb4ae4b5ccbec174139142)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:04.936102 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daff634-b343-4702-b26b-70a05e5bf1a4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:04.936387 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 188/377] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 7 07:42:04 2022] GET /identity/v3/groups/a06ef62241eb4ae4b5ccbec174139142 => generated 226 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:04.941898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daff634-b343-4702-b26b-70a05e5bf1a4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:04.984744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:05.061695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4929959-6b01-4333-b9e2-2cca87f46df2 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:05.133594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4321cf-aa74-43d5-9fdf-95ca40f66863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:05.136013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daff634-b343-4702-b26b-70a05e5bf1a4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:05.136878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4321cf-aa74-43d5-9fdf-95ca40f66863 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:05.137390 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4321cf-aa74-43d5-9fdf-95ca40f66863 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:05.137872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4321cf-aa74-43d5-9fdf-95ca40f66863 None admin] PATH_INFO: `/v3/groups/1782e6af28b34561884d3ba44a55628b` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:05.152342 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4321cf-aa74-43d5-9fdf-95ca40f66863 None admin] RBAC: Authorizing `identity:get_group(group_id=1782e6af28b34561884d3ba44a55628b)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:05.154111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4321cf-aa74-43d5-9fdf-95ca40f66863 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:05.157588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 190/378] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 7 07:42:04 2022] GET /identity/v3/groups/1782e6af28b34561884d3ba44a55628b => generated 229 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:05.178703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4321cf-aa74-43d5-9fdf-95ca40f66863 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:05.211859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470b992-0d99-4f48-9fd3-9ad2611e3c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:05.212836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470b992-0d99-4f48-9fd3-9ad2611e3c23 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:05.213325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470b992-0d99-4f48-9fd3-9ad2611e3c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:05.213810 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470b992-0d99-4f48-9fd3-9ad2611e3c23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:05.214736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470b992-0d99-4f48-9fd3-9ad2611e3c23 None admin] RBAC: Adding query filter params (group.id=a06ef62241eb4ae4b5ccbec174139142, role.id=84a30faea3e94eb48b03ec7ceb7c1392, scope.project.id=ea579b172f214888a95d9af5a4e1d122) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:05.216341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470b992-0d99-4f48-9fd3-9ad2611e3c23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:05.217982 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470b992-0d99-4f48-9fd3-9ad2611e3c23 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:05.230534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 189/379] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 7 07:42:04 2022] GET /identity/v3/role_assignments?group.id=a06ef62241eb4ae4b5ccbec174139142&scope.project.id=ea579b172f214888a95d9af5a4e1d122&role.id=84a30faea3e94eb48b03ec7ceb7c1392 => generated 260 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:05.279043 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:05.340709 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4321cf-aa74-43d5-9fdf-95ca40f66863 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:05.382954 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538e209-3cfb-49fd-b16a-4bfbe42ce178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:05.388102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538e209-3cfb-49fd-b16a-4bfbe42ce178 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:05.388102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538e209-3cfb-49fd-b16a-4bfbe42ce178 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:05.388102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538e209-3cfb-49fd-b16a-4bfbe42ce178 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:05.388102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538e209-3cfb-49fd-b16a-4bfbe42ce178 None admin] RBAC: Adding query filter params (group.id=1782e6af28b34561884d3ba44a55628b, role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=bb18598c40b6447faef02991e1b9c4ed) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:05.388102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538e209-3cfb-49fd-b16a-4bfbe42ce178 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:05.388102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538e209-3cfb-49fd-b16a-4bfbe42ce178 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:05.396958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 191/380] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 7 07:42:05 2022] GET /identity/v3/role_assignments?group.id=1782e6af28b34561884d3ba44a55628b&scope.project.id=bb18598c40b6447faef02991e1b9c4ed&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:05.443240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac1846-7fc7-45e6-a9c3-67f4f66d28ed None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:05.443744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac1846-7fc7-45e6-a9c3-67f4f66d28ed None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:05.443744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac1846-7fc7-45e6-a9c3-67f4f66d28ed None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:05.445428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 190/381] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:05.451498 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff4334-5942-4d81-9047-9b1a8c661c5a None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:05.451815 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff4334-5942-4d81-9047-9b1a8c661c5a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:05.452054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff4334-5942-4d81-9047-9b1a8c661c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:05.481296 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-cfff4334-5942-4d81-9047-9b1a8c661c5a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:05.483007 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cfff4334-5942-4d81-9047-9b1a8c661c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:05.582620 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 192/382] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:05.891112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac1846-7fc7-45e6-a9c3-67f4f66d28ed None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:05.998928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:06.020805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac1846-7fc7-45e6-a9c3-67f4f66d28ed None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:06.059659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3b8aa-4b2b-40ac-81fe-9d0371904aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:06.060665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3b8aa-4b2b-40ac-81fe-9d0371904aeb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:06.061146 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3b8aa-4b2b-40ac-81fe-9d0371904aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:06.061644 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3b8aa-4b2b-40ac-81fe-9d0371904aeb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:06.074918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3b8aa-4b2b-40ac-81fe-9d0371904aeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:06.080822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-77d3b8aa-4b2b-40ac-81fe-9d0371904aeb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:42:06.080822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:06.080822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:06.080822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:06.080822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:06.080822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:06.080822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:06.080822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:06.080822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:06.080822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:06.080822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:06.080822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:06.080822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:06.080822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:06.080822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:06.080822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:06.081791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:06.081791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:06.081791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:06.081791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:06.081791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:06.081791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:06.081791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:06.081791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:06.081791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:06.081791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:06.081791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:06.081791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:06.081791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:06.081791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:06.081791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:06.081791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:06.082566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:06.082566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:06.082566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:06.082566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:06.082566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:06.082566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:06.082566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:06.082566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:06.082566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:06.082566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:06.082566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:42:06.082566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:06.085543 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 191/383] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:06.091460 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff4334-5942-4d81-9047-9b1a8c661c5a None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:06.174101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:06.233987 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff4334-5942-4d81-9047-9b1a8c661c5a None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:06.277558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc43cf4-6e69-443d-b07b-814c020da98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:06.278516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc43cf4-6e69-443d-b07b-814c020da98c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:06.278995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc43cf4-6e69-443d-b07b-814c020da98c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:06.279487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc43cf4-6e69-443d-b07b-814c020da98c None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:06.280782 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc43cf4-6e69-443d-b07b-814c020da98c None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:06.281300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc43cf4-6e69-443d-b07b-814c020da98c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:06.282946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc43cf4-6e69-443d-b07b-814c020da98c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:06.295964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 193/384] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:42:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:06.302849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3b8aa-4b2b-40ac-81fe-9d0371904aeb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:06.438787 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3b8aa-4b2b-40ac-81fe-9d0371904aeb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:06.482204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd576ed9-99b4-4111-a16c-89198b32ebf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:06.483628 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd576ed9-99b4-4111-a16c-89198b32ebf1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:06.484212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd576ed9-99b4-4111-a16c-89198b32ebf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:06.484703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd576ed9-99b4-4111-a16c-89198b32ebf1 None admin] PATH_INFO: `/v3/groups/a06ef62241eb4ae4b5ccbec174139142` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:06.488754 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd576ed9-99b4-4111-a16c-89198b32ebf1 None admin] RBAC: Authorizing `identity:get_group(group_id=a06ef62241eb4ae4b5ccbec174139142)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:06.490743 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd576ed9-99b4-4111-a16c-89198b32ebf1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:06.492757 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 192/385] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 7 07:42:06 2022] GET /identity/v3/groups/a06ef62241eb4ae4b5ccbec174139142 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:06.498452 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc43cf4-6e69-443d-b07b-814c020da98c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:06.614495 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5d32a-408b-4d44-aec2-0fbbfebf5eef None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:06.615145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5d32a-408b-4d44-aec2-0fbbfebf5eef None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:06.615623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5d32a-408b-4d44-aec2-0fbbfebf5eef None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:06.617185 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 193/386] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:42:06.627387 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f8657-dbff-45a1-962f-63f0d1d92571 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:06.627926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f8657-dbff-45a1-962f-63f0d1d92571 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:06.628455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f8657-dbff-45a1-962f-63f0d1d92571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:06.653612 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1e6f8657-dbff-45a1-962f-63f0d1d92571 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:06.655821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1e6f8657-dbff-45a1-962f-63f0d1d92571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:06.659465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc43cf4-6e69-443d-b07b-814c020da98c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:06.674355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:06.700529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:06.723140 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc12bd-58d9-444e-a537-b4fc1c824fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:06.723775 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc12bd-58d9-444e-a537-b4fc1c824fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:06.724055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc12bd-58d9-444e-a537-b4fc1c824fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:06.724366 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc12bd-58d9-444e-a537-b4fc1c824fd5 None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:06.725899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc12bd-58d9-444e-a537-b4fc1c824fd5 None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:06.727781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc12bd-58d9-444e-a537-b4fc1c824fd5 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:06.734168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 194/387] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:42:06 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:06.735027 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc12bd-58d9-444e-a537-b4fc1c824fd5 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:06.791883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 194/388] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:42:06.893569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc12bd-58d9-444e-a537-b4fc1c824fd5 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:06.948858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cadc1b-7c58-4e50-b5d1-65b7fb4142f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:06.949535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cadc1b-7c58-4e50-b5d1-65b7fb4142f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:06.949814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cadc1b-7c58-4e50-b5d1-65b7fb4142f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:06.950090 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cadc1b-7c58-4e50-b5d1-65b7fb4142f0 None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5/groups/a06ef62241eb4ae4b5ccbec174139142/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:06.957631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cadc1b-7c58-4e50-b5d1-65b7fb4142f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5, group_id=a06ef62241eb4ae4b5ccbec174139142, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:06.959036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cadc1b-7c58-4e50-b5d1-65b7fb4142f0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:06.981050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 195/389] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Sep 7 07:42:06 2022] PUT /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5/groups/a06ef62241eb4ae4b5ccbec174139142/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:42:07.054620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367be7b-893c-447d-a6a2-70cf4f55c215 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:07.054960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367be7b-893c-447d-a6a2-70cf4f55c215 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:07.055183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367be7b-893c-447d-a6a2-70cf4f55c215 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:07.056720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 195/390] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:07.065976 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62327147-b7cc-412d-a125-63490256536d None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:07.066255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62327147-b7cc-412d-a125-63490256536d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:07.066537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62327147-b7cc-412d-a125-63490256536d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:07.095907 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-62327147-b7cc-412d-a125-63490256536d None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:07.097525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-62327147-b7cc-412d-a125-63490256536d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:07.148449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367be7b-893c-447d-a6a2-70cf4f55c215 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:07.151832 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:07.181262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 196/391] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:07.311387 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367be7b-893c-447d-a6a2-70cf4f55c215 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:07.349018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66dd246-c015-4086-b60e-c1f19fca6f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:07.349649 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dd246-c015-4086-b60e-c1f19fca6f43 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:07.349976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dd246-c015-4086-b60e-c1f19fca6f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:07.350293 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dd246-c015-4086-b60e-c1f19fca6f43 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:07.361811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66dd246-c015-4086-b60e-c1f19fca6f43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:07.366861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-c66dd246-c015-4086-b60e-c1f19fca6f43 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:42:07.366861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:07.366861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:07.366861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:07.366861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:07.366861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:07.366861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:07.366861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:07.366861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:07.366861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:07.366861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:07.366861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:07.366861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:07.366861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:07.366861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:07.366861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:07.367722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:07.367722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:07.367722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:07.367722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:07.367722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:07.367722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:07.367722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:07.367722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:07.367722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:07.367722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:07.367722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:07.367722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:07.367722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:07.367722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:07.367722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:07.367722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:07.368544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:07.368544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:07.368544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:07.368544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:07.368544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:07.368544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:07.368544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:07.368544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:07.368544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:07.368544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:07.368544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:42:07.368544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:07.368544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 196/392] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:42:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:07.374089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62327147-b7cc-412d-a125-63490256536d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:07.483717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66dd246-c015-4086-b60e-c1f19fca6f43 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:07.487819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:07.518674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62327147-b7cc-412d-a125-63490256536d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:07.573088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c2cca-351e-4b7c-8455-a294b2f3f117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:07.573620 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c2cca-351e-4b7c-8455-a294b2f3f117 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:07.573894 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c2cca-351e-4b7c-8455-a294b2f3f117 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:07.574160 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c2cca-351e-4b7c-8455-a294b2f3f117 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:07.574817 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c2cca-351e-4b7c-8455-a294b2f3f117 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:07.575110 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c2cca-351e-4b7c-8455-a294b2f3f117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:07.579096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c2cca-351e-4b7c-8455-a294b2f3f117 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:07.600627 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 197/393] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:07.606819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c2cca-351e-4b7c-8455-a294b2f3f117 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:07.651686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66dd246-c015-4086-b60e-c1f19fca6f43 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:07.691546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:07.730279 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8551be1-2bf9-4154-95f3-dd77731be4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:07.730912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8551be1-2bf9-4154-95f3-dd77731be4a0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:07.731187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8551be1-2bf9-4154-95f3-dd77731be4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:07.731450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8551be1-2bf9-4154-95f3-dd77731be4a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:07.745830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8551be1-2bf9-4154-95f3-dd77731be4a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:07.752490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-a8551be1-2bf9-4154-95f3-dd77731be4a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:42:07.752490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:07.752490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:07.752490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:07.752490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:07.752490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:07.752490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:07.752490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:07.752490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:07.752490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:07.752490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:07.752490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:07.752490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:07.752490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:07.752490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:07.752490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:07.753399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:07.753399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:07.753399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:07.753399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:07.753399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:07.753399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:07.753399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:07.753399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:07.753399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:07.753399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:07.753399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:07.753399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:07.753399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:07.753399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:07.753399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:07.753399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:07.754588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:07.754588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:07.754588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:07.754588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:07.754588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:07.754588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:07.754588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:07.754588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:07.754588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:07.754588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:07.754588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:42:07.754588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:07.754588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 197/394] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:42:07.761107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8551be1-2bf9-4154-95f3-dd77731be4a0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:07.808407 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c2cca-351e-4b7c-8455-a294b2f3f117 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:07.874907 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e84f50-f0bc-460d-8159-0880b071e056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:07.875473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e84f50-f0bc-460d-8159-0880b071e056 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:07.876339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e84f50-f0bc-460d-8159-0880b071e056 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:07.876668 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e84f50-f0bc-460d-8159-0880b071e056 None admin] PATH_INFO: `/v3/groups/a06ef62241eb4ae4b5ccbec174139142` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:07.878119 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e84f50-f0bc-460d-8159-0880b071e056 None admin] RBAC: Authorizing `identity:get_group(group_id=a06ef62241eb4ae4b5ccbec174139142)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:07.879750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e84f50-f0bc-460d-8159-0880b071e056 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:07.880747 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:07.883529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 198/395] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 7 07:42:07 2022] GET /identity/v3/groups/a06ef62241eb4ae4b5ccbec174139142 => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:07.889539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e84f50-f0bc-460d-8159-0880b071e056 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:07.964216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8551be1-2bf9-4154-95f3-dd77731be4a0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:08.012994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0540293-2f44-4327-956d-642af3d85290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:08.013543 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0540293-2f44-4327-956d-642af3d85290 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:08.013818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0540293-2f44-4327-956d-642af3d85290 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:08.014088 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0540293-2f44-4327-956d-642af3d85290 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:08.014745 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0540293-2f44-4327-956d-642af3d85290 None admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:08.015043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0540293-2f44-4327-956d-642af3d85290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:08.016436 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0540293-2f44-4327-956d-642af3d85290 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:08.026447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 198/396] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:42:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:08.027709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ec90d-371d-4ba9-8240-07f4f6edf164 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:08.027991 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ec90d-371d-4ba9-8240-07f4f6edf164 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:08.028294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ec90d-371d-4ba9-8240-07f4f6edf164 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:08.033707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 199/397] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:08.034646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ec90d-371d-4ba9-8240-07f4f6edf164 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:08.054277 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e84f50-f0bc-460d-8159-0880b071e056 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:08.062663 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:08.094103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f7054-6e47-49ca-a3f4-d4adf104121e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:08.094845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f7054-6e47-49ca-a3f4-d4adf104121e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:08.095152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f7054-6e47-49ca-a3f4-d4adf104121e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:08.095677 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f7054-6e47-49ca-a3f4-d4adf104121e None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:08.096974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f7054-6e47-49ca-a3f4-d4adf104121e None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:08.098447 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f7054-6e47-49ca-a3f4-d4adf104121e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:08.100520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 199/398] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:42:07 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:08.101651 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ff0ca-cf54-4f1b-9267-93cc629da51f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:08.101924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ff0ca-cf54-4f1b-9267-93cc629da51f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:08.102210 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ff0ca-cf54-4f1b-9267-93cc629da51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:08.121890 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-400ff0ca-cf54-4f1b-9267-93cc629da51f None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:08.123078 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-400ff0ca-cf54-4f1b-9267-93cc629da51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:08.167868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ec90d-371d-4ba9-8240-07f4f6edf164 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:08.194827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:08.204246 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 200/399] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:08.204246 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ff0ca-cf54-4f1b-9267-93cc629da51f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:08.215067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4a592-4c71-4fde-baa7-aaf634c6dffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:08.215854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4a592-4c71-4fde-baa7-aaf634c6dffd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:08.216174 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4a592-4c71-4fde-baa7-aaf634c6dffd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:08.216452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4a592-4c71-4fde-baa7-aaf634c6dffd None admin] PATH_INFO: `/v3/groups/a06ef62241eb4ae4b5ccbec174139142` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:08.217719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4a592-4c71-4fde-baa7-aaf634c6dffd None admin] RBAC: Authorizing `identity:get_group(group_id=a06ef62241eb4ae4b5ccbec174139142)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:08.219294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4a592-4c71-4fde-baa7-aaf634c6dffd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:08.225822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 200/400] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 7 07:42:08 2022] GET /identity/v3/groups/a06ef62241eb4ae4b5ccbec174139142 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:08.227194 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c1f55-59ae-417e-b021-df98c3421342 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:08.227527 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c1f55-59ae-417e-b021-df98c3421342 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:08.227796 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c1f55-59ae-417e-b021-df98c3421342 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:08.229260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 201/401] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:08.229977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c1f55-59ae-417e-b021-df98c3421342 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:08.325828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ff0ca-cf54-4f1b-9267-93cc629da51f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:08.328529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:08.350525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c1f55-59ae-417e-b021-df98c3421342 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:08.371318 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da33cf-2d72-4fa6-ba1a-ef834e92d195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:08.372669 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da33cf-2d72-4fa6-ba1a-ef834e92d195 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:08.373397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da33cf-2d72-4fa6-ba1a-ef834e92d195 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:08.373897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da33cf-2d72-4fa6-ba1a-ef834e92d195 None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5/groups/a06ef62241eb4ae4b5ccbec174139142/roles/84a30faea3e94eb48b03ec7ceb7c1392` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:08.377530 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da33cf-2d72-4fa6-ba1a-ef834e92d195 None admin] RBAC: Authorizing `identity:create_grant(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5, group_id=a06ef62241eb4ae4b5ccbec174139142, role_id=84a30faea3e94eb48b03ec7ceb7c1392)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:08.380308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da33cf-2d72-4fa6-ba1a-ef834e92d195 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:08.387558 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1c252-5e8b-404e-876d-590a9647356c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:08.389171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1c252-5e8b-404e-876d-590a9647356c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:08.389171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1c252-5e8b-404e-876d-590a9647356c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:08.389521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1c252-5e8b-404e-876d-590a9647356c None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:08.390454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1c252-5e8b-404e-876d-590a9647356c None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:08.392292 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1c252-5e8b-404e-876d-590a9647356c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:08.399231 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 202/402] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:42:08 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:08.400531 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c77030-009f-4b2a-a9ff-4e803c4fe197 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:08.400812 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c77030-009f-4b2a-a9ff-4e803c4fe197 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:08.401119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c77030-009f-4b2a-a9ff-4e803c4fe197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:08.401571 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 201/403] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Sep 7 07:42:08 2022] PUT /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5/groups/a06ef62241eb4ae4b5ccbec174139142/roles/84a30faea3e94eb48b03ec7ceb7c1392 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:42:08.402617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da33cf-2d72-4fa6-ba1a-ef834e92d195 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:08.434336 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e3c77030-009f-4b2a-a9ff-4e803c4fe197 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:08.435475 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e3c77030-009f-4b2a-a9ff-4e803c4fe197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:08.534158 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da33cf-2d72-4fa6-ba1a-ef834e92d195 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:08.549182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:08.572119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 203/404] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:08.572119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c77030-009f-4b2a-a9ff-4e803c4fe197 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:08.596988 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b942072-ca05-4775-b2f1-dfb14346d888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:08.597900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b942072-ca05-4775-b2f1-dfb14346d888 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:08.598174 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b942072-ca05-4775-b2f1-dfb14346d888 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:08.598443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b942072-ca05-4775-b2f1-dfb14346d888 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122/groups/a06ef62241eb4ae4b5ccbec174139142/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:08.603712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b942072-ca05-4775-b2f1-dfb14346d888 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea579b172f214888a95d9af5a4e1d122, group_id=a06ef62241eb4ae4b5ccbec174139142, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:08.605254 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b942072-ca05-4775-b2f1-dfb14346d888 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:08.634517 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 202/405] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Sep 7 07:42:08 2022] PUT /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122/groups/a06ef62241eb4ae4b5ccbec174139142/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:42:08.707225 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c77030-009f-4b2a-a9ff-4e803c4fe197 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:08.746784 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515acca-58ef-42ed-babc-a1470cf2499d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:08.747496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515acca-58ef-42ed-babc-a1470cf2499d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:08.747773 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515acca-58ef-42ed-babc-a1470cf2499d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:08.748171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515acca-58ef-42ed-babc-a1470cf2499d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:08.760603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515acca-58ef-42ed-babc-a1470cf2499d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:08.765637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-2515acca-58ef-42ed-babc-a1470cf2499d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:42:08.765637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:08.765637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:08.765637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:08.765637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:08.765637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:08.765637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:08.765637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:08.765637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:08.765637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:08.765637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:08.765637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:08.765637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:08.765637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:08.765637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:08.765637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:08.766581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:08.766581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:08.766581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:08.766581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:08.766581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:08.766581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:08.766581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:08.766581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:08.766581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:08.766581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:08.766581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:08.766581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:08.766581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:08.766581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:08.766581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:08.766581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:08.773043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:08.773043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:08.773043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:08.773043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:08.773043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:08.773043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:08.773043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:08.773043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:08.773043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:08.773043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:08.773043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:42:08.773043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:08.773043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 204/406] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:42:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:08.773787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b942072-ca05-4775-b2f1-dfb14346d888 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:08.802302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:08.895092 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b942072-ca05-4775-b2f1-dfb14346d888 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:08.910771 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515acca-58ef-42ed-babc-a1470cf2499d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:08.941485 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5d967-0978-4415-b606-d6ffa3d2d8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:08.942190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5d967-0978-4415-b606-d6ffa3d2d8fa None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:08.942473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5d967-0978-4415-b606-d6ffa3d2d8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:08.942741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5d967-0978-4415-b606-d6ffa3d2d8fa None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:08.943396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5d967-0978-4415-b606-d6ffa3d2d8fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:08.943696 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5d967-0978-4415-b606-d6ffa3d2d8fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:08.947400 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5d967-0978-4415-b606-d6ffa3d2d8fa None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:08.957797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 203/407] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:08.965182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5d967-0978-4415-b606-d6ffa3d2d8fa None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:08.977203 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:09.034870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515acca-58ef-42ed-babc-a1470cf2499d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:09.075951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a275b0-48bd-417c-b8c0-1f6d87564072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:09.076608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a275b0-48bd-417c-b8c0-1f6d87564072 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:09.076880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a275b0-48bd-417c-b8c0-1f6d87564072 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:09.077144 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a275b0-48bd-417c-b8c0-1f6d87564072 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:09.117044 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a275b0-48bd-417c-b8c0-1f6d87564072 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:09.121946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-37a275b0-48bd-417c-b8c0-1f6d87564072 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:42:09.121946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:09.121946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:09.121946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:09.121946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:09.121946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:09.121946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:09.121946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:09.121946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:09.121946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:09.121946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:09.121946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:09.121946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:09.121946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:09.121946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:09.121946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:09.122803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:09.122803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:09.122803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:09.122803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:09.122803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:09.122803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:09.122803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:09.122803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:09.122803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:09.122803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:09.122803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:09.122803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:09.122803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:09.122803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:09.122803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:09.122803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:09.123610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:09.123610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:09.123610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:09.123610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:09.123610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:09.123610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:09.123610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:09.123610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:09.123610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:09.123610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:09.123610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:42:09.123610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:09.123610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 205/408] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:42:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:42:09.128138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:09.133165 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a275b0-48bd-417c-b8c0-1f6d87564072 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:09.145187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5d967-0978-4415-b606-d6ffa3d2d8fa None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:09.192658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe2af1-8432-4547-957f-bd6136e307a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:09.194565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe2af1-8432-4547-957f-bd6136e307a3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:09.195941 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe2af1-8432-4547-957f-bd6136e307a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:09.196391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe2af1-8432-4547-957f-bd6136e307a3 None admin] PATH_INFO: `/v3/groups/1782e6af28b34561884d3ba44a55628b` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:09.198440 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe2af1-8432-4547-957f-bd6136e307a3 None admin] RBAC: Authorizing `identity:get_group(group_id=1782e6af28b34561884d3ba44a55628b)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:09.200108 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe2af1-8432-4547-957f-bd6136e307a3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:09.203451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 204/409] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 7 07:42:08 2022] GET /identity/v3/groups/1782e6af28b34561884d3ba44a55628b => generated 229 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:09.210603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe2af1-8432-4547-957f-bd6136e307a3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:09.334454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a275b0-48bd-417c-b8c0-1f6d87564072 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:09.358461 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:09.402074 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300f11c-5039-4bce-9a56-338590589ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:09.402906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300f11c-5039-4bce-9a56-338590589ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:09.403185 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300f11c-5039-4bce-9a56-338590589ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:09.403452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300f11c-5039-4bce-9a56-338590589ca7 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:09.405567 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe2af1-8432-4547-957f-bd6136e307a3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:09.407340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300f11c-5039-4bce-9a56-338590589ca7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:09.407735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300f11c-5039-4bce-9a56-338590589ca7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:09.409358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300f11c-5039-4bce-9a56-338590589ca7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:09.427286 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 206/410] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:09.434031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300f11c-5039-4bce-9a56-338590589ca7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:09.473938 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa6177-604f-4985-93ff-3a5503351191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:09.474542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa6177-604f-4985-93ff-3a5503351191 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:09.474818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa6177-604f-4985-93ff-3a5503351191 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:09.475085 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa6177-604f-4985-93ff-3a5503351191 None admin] PATH_INFO: `/v3/projects/bb18598c40b6447faef02991e1b9c4ed` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:09.477340 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa6177-604f-4985-93ff-3a5503351191 None admin] RBAC: Authorizing `identity:get_project(project_id=bb18598c40b6447faef02991e1b9c4ed)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:09.479563 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa6177-604f-4985-93ff-3a5503351191 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:09.482353 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 205/411] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:42:09 2022] GET /identity/v3/projects/bb18598c40b6447faef02991e1b9c4ed => generated 344 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:09.488370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa6177-604f-4985-93ff-3a5503351191 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:09.531724 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:09.616547 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300f11c-5039-4bce-9a56-338590589ca7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:09.681635 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa6177-604f-4985-93ff-3a5503351191 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:09.693328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6105874-01b8-43a7-93df-fb7b9e61214a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:09.693964 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6105874-01b8-43a7-93df-fb7b9e61214a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:09.694506 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6105874-01b8-43a7-93df-fb7b9e61214a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:09.694777 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6105874-01b8-43a7-93df-fb7b9e61214a None admin] PATH_INFO: `/v3/groups/a06ef62241eb4ae4b5ccbec174139142` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:09.700750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6105874-01b8-43a7-93df-fb7b9e61214a None admin] RBAC: Authorizing `identity:get_group(group_id=a06ef62241eb4ae4b5ccbec174139142)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:09.702176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6105874-01b8-43a7-93df-fb7b9e61214a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:09.703742 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 207/412] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 7 07:42:09 2022] GET /identity/v3/groups/a06ef62241eb4ae4b5ccbec174139142 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:09.710000 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6105874-01b8-43a7-93df-fb7b9e61214a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:09.719026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283bbe15-0cde-46bf-9729-18961171c8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:09.719734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bbe15-0cde-46bf-9729-18961171c8eb None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:09.720012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bbe15-0cde-46bf-9729-18961171c8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:09.720570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bbe15-0cde-46bf-9729-18961171c8eb None admin] PATH_INFO: `/v3/projects/bb18598c40b6447faef02991e1b9c4ed/groups/1782e6af28b34561884d3ba44a55628b/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:09.722607 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283bbe15-0cde-46bf-9729-18961171c8eb None admin] RBAC: Authorizing `identity:create_grant(project_id=bb18598c40b6447faef02991e1b9c4ed, group_id=1782e6af28b34561884d3ba44a55628b, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:09.724153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283bbe15-0cde-46bf-9729-18961171c8eb None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:09.741087 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 206/413] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Sep 7 07:42:09 2022] PUT /identity/v3/projects/bb18598c40b6447faef02991e1b9c4ed/groups/1782e6af28b34561884d3ba44a55628b/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:42:09.785377 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:09.793996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0fdf4-b2b4-47f4-9c66-909c8f739142 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:09.794297 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0fdf4-b2b4-47f4-9c66-909c8f739142 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:09.794597 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0fdf4-b2b4-47f4-9c66-909c8f739142 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:09.796105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 207/414] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:09.801384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced24dd-b76c-44e9-a9aa-2531f36bbfa0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:09.801912 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced24dd-b76c-44e9-a9aa-2531f36bbfa0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:09.802248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced24dd-b76c-44e9-a9aa-2531f36bbfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:09.823535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9ced24dd-b76c-44e9-a9aa-2531f36bbfa0 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:09.824810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9ced24dd-b76c-44e9-a9aa-2531f36bbfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:09.895519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6105874-01b8-43a7-93df-fb7b9e61214a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:09.904391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 208/415] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:42:09.963477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdc244-381f-4e90-a4b7-e3bd2c28210f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:09.964575 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdc244-381f-4e90-a4b7-e3bd2c28210f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:09.965048 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdc244-381f-4e90-a4b7-e3bd2c28210f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:09.965514 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdc244-381f-4e90-a4b7-e3bd2c28210f None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:09.972993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fdc244-381f-4e90-a4b7-e3bd2c28210f None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:09.974892 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fdc244-381f-4e90-a4b7-e3bd2c28210f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:09.976920 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 208/416] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:42:09 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:09.983045 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced24dd-b76c-44e9-a9aa-2531f36bbfa0 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:10.007240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:10.104336 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced24dd-b76c-44e9-a9aa-2531f36bbfa0 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:10.140634 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec6113-c7c8-4b7c-934a-88019bc897b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:10.141295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec6113-c7c8-4b7c-934a-88019bc897b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:10.141567 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec6113-c7c8-4b7c-934a-88019bc897b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:10.141832 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec6113-c7c8-4b7c-934a-88019bc897b3 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122/groups/a06ef62241eb4ae4b5ccbec174139142/roles/84a30faea3e94eb48b03ec7ceb7c1392` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:10.144757 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec6113-c7c8-4b7c-934a-88019bc897b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea579b172f214888a95d9af5a4e1d122, group_id=a06ef62241eb4ae4b5ccbec174139142, role_id=84a30faea3e94eb48b03ec7ceb7c1392)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:10.146187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec6113-c7c8-4b7c-934a-88019bc897b3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:10.165525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 209/417] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Sep 7 07:42:09 2022] PUT /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122/groups/a06ef62241eb4ae4b5ccbec174139142/roles/84a30faea3e94eb48b03ec7ceb7c1392 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:42:10.206744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdc244-381f-4e90-a4b7-e3bd2c28210f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:10.370224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdc244-381f-4e90-a4b7-e3bd2c28210f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:10.380483 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:10.409616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa0a79-1806-4edd-a86e-f7b829084870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:10.410267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa0a79-1806-4edd-a86e-f7b829084870 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:10.410545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa0a79-1806-4edd-a86e-f7b829084870 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:10.410819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa0a79-1806-4edd-a86e-f7b829084870 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:10.422940 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa0a79-1806-4edd-a86e-f7b829084870 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:10.428422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ddfa0a79-1806-4edd-a86e-f7b829084870 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:42:10.428422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:10.428422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:10.428422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:10.428422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:10.428422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:10.428422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:10.428422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:10.428422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:10.428422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:10.428422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:10.428422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:10.428422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:10.428422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:10.428422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:10.428422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:10.429319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:10.429319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:10.429319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:10.429319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:10.429319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:10.429319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:10.429319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:10.429319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:10.429319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:10.429319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:10.429319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:10.429319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:10.429319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:10.429319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:10.429319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:10.429319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:10.430133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:10.430133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:10.430133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:10.430133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:10.430133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:10.430133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:10.430133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:10.430133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:10.430133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:10.430133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:10.430133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:42:10.430133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:10.430133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 209/418] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:10.435695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec6113-c7c8-4b7c-934a-88019bc897b3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:10.512423 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:10.545487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec6113-c7c8-4b7c-934a-88019bc897b3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:10.585320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292fd5d-2cf0-4da9-bca7-3e1ddb3054d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:10.585928 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292fd5d-2cf0-4da9-bca7-3e1ddb3054d1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:10.586205 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292fd5d-2cf0-4da9-bca7-3e1ddb3054d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:10.586473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292fd5d-2cf0-4da9-bca7-3e1ddb3054d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:10.587125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292fd5d-2cf0-4da9-bca7-3e1ddb3054d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:10.587424 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292fd5d-2cf0-4da9-bca7-3e1ddb3054d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:10.588826 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292fd5d-2cf0-4da9-bca7-3e1ddb3054d1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:10.598303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 210/419] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:42:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:10.603681 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa0a79-1806-4edd-a86e-f7b829084870 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:10.776504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa0a79-1806-4edd-a86e-f7b829084870 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:10.835162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca8867-e143-459f-8ab8-875dcee90413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:10.836152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca8867-e143-459f-8ab8-875dcee90413 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:10.836626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca8867-e143-459f-8ab8-875dcee90413 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:10.837113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca8867-e143-459f-8ab8-875dcee90413 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:10.881668 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:10.923932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca8867-e143-459f-8ab8-875dcee90413 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:10.925609 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca8867-e143-459f-8ab8-875dcee90413 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:11.070669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 210/420] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:11.078970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292fd5d-2cf0-4da9-bca7-3e1ddb3054d1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:11.200410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292fd5d-2cf0-4da9-bca7-3e1ddb3054d1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:11.234681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d070e-0463-4bff-8cec-6153b09d7589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:11.235312 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d070e-0463-4bff-8cec-6153b09d7589 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:11.235591 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d070e-0463-4bff-8cec-6153b09d7589 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:11.236467 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d070e-0463-4bff-8cec-6153b09d7589 None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:11.238069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d070e-0463-4bff-8cec-6153b09d7589 None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:11.239514 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d070e-0463-4bff-8cec-6153b09d7589 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:11.241536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 211/421] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:42:11 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:11.248516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca8867-e143-459f-8ab8-875dcee90413 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:11.264718 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e231c8c-1f31-4253-8c29-81e4391bc8b8 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:11.265019 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e231c8c-1f31-4253-8c29-81e4391bc8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:11.265266 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e231c8c-1f31-4253-8c29-81e4391bc8b8 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:11.266846 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 212/422] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:11.271810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d94b66-08cb-4843-99e4-e96b093d94f6 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:11.272104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d94b66-08cb-4843-99e4-e96b093d94f6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:11.272388 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d94b66-08cb-4843-99e4-e96b093d94f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:11.278678 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:11.309187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-86d94b66-08cb-4843-99e4-e96b093d94f6 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:11.310404 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-86d94b66-08cb-4843-99e4-e96b093d94f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:11.378373 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:11.399435 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 213/423] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:42:11.429345 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca8867-e143-459f-8ab8-875dcee90413 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:11.477231 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac842e4a-7255-4835-8f93-ec356a7beb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:11.477863 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac842e4a-7255-4835-8f93-ec356a7beb7a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:11.478141 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac842e4a-7255-4835-8f93-ec356a7beb7a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:11.478465 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac842e4a-7255-4835-8f93-ec356a7beb7a None admin] PATH_INFO: `/v3/groups/a06ef62241eb4ae4b5ccbec174139142` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:11.480684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac842e4a-7255-4835-8f93-ec356a7beb7a None admin] RBAC: Authorizing `identity:get_group(group_id=a06ef62241eb4ae4b5ccbec174139142)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:11.482120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac842e4a-7255-4835-8f93-ec356a7beb7a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:11.483995 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 211/424] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 7 07:42:11 2022] GET /identity/v3/groups/a06ef62241eb4ae4b5ccbec174139142 => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:11.490029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d94b66-08cb-4843-99e4-e96b093d94f6 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:11.515122 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5aa42-af09-4a81-9def-3d9d94304efd None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:11.515707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5aa42-af09-4a81-9def-3d9d94304efd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:11.516182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5aa42-af09-4a81-9def-3d9d94304efd None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:11.518154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 212/425] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:42:11.523238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5105822-fbcb-4c63-9ea2-b72a7d960ce8 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:11.523718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5105822-fbcb-4c63-9ea2-b72a7d960ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:11.524195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5105822-fbcb-4c63-9ea2-b72a7d960ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:11.542761 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e5105822-fbcb-4c63-9ea2-b72a7d960ce8 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:11.544256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e5105822-fbcb-4c63-9ea2-b72a7d960ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:11.625272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d94b66-08cb-4843-99e4-e96b093d94f6 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:11.676466 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:11.685788 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821d922-ae4c-4cd7-8c40-fde709723bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:11.686389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821d922-ae4c-4cd7-8c40-fde709723bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:11.687526 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821d922-ae4c-4cd7-8c40-fde709723bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:11.687803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821d922-ae4c-4cd7-8c40-fde709723bd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:11.688550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821d922-ae4c-4cd7-8c40-fde709723bd1 None admin] RBAC: Adding query filter params (group.id=a06ef62241eb4ae4b5ccbec174139142, role.id=6ebbed85905048a198e4261a3510dafc, scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:11.689409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821d922-ae4c-4cd7-8c40-fde709723bd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:11.690964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821d922-ae4c-4cd7-8c40-fde709723bd1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:11.702685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 213/426] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:42:11.705228 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 214/427] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 7 07:42:11 2022] GET /identity/v3/role_assignments?group.id=a06ef62241eb4ae4b5ccbec174139142&scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5&role.id=6ebbed85905048a198e4261a3510dafc => generated 611 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:11.918556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5105822-fbcb-4c63-9ea2-b72a7d960ce8 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:11.979019 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821d922-ae4c-4cd7-8c40-fde709723bd1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:11.994734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:12.044125 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5105822-fbcb-4c63-9ea2-b72a7d960ce8 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:12.096622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85446c-d2ee-4a8d-bd09-e7028bfffc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:12.097328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85446c-d2ee-4a8d-bd09-e7028bfffc56 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:12.098205 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85446c-d2ee-4a8d-bd09-e7028bfffc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:12.098205 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85446c-d2ee-4a8d-bd09-e7028bfffc56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:12.114755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85446c-d2ee-4a8d-bd09-e7028bfffc56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:12.121439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-0f85446c-d2ee-4a8d-bd09-e7028bfffc56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:42:12.121439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:12.121439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:12.121439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:12.121439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:12.121439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:12.121439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:12.121439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:12.121439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:12.121439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:12.121439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:12.121439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:12.121439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:12.121439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:12.121439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:12.121439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:12.122327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:12.122327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:12.122327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:12.122327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:12.122327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:12.122327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:12.122327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:12.122327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:12.122327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:12.122327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:12.122327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:12.122327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:12.122327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:12.122327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:12.122327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:12.122327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:12.123134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:12.123134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:12.123134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:12.123134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:12.123134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:12.123134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:12.123134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:12.123134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:12.123134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:12.123134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:12.123134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 07:42:12.123134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:12.123134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 214/428] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:12.128379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85446c-d2ee-4a8d-bd09-e7028bfffc56 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:12.136983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821d922-ae4c-4cd7-8c40-fde709723bd1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:12.177872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23980b-d5e5-4a19-b6cc-ec2b4efb6f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:12.178907 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23980b-d5e5-4a19-b6cc-ec2b4efb6f56 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:12.179366 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23980b-d5e5-4a19-b6cc-ec2b4efb6f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:12.179937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23980b-d5e5-4a19-b6cc-ec2b4efb6f56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:12.197379 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23980b-d5e5-4a19-b6cc-ec2b4efb6f56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:12.203953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-6e23980b-d5e5-4a19-b6cc-ec2b4efb6f56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:42:12.203953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:12.203953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:12.203953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:12.203953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:12.203953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:12.203953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:12.203953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:12.203953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:12.203953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:12.203953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:12.203953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:12.203953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:12.203953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:12.203953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:12.203953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:12.205387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:12.205387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:12.205387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:12.205387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:12.205387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:12.205387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:12.205387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:12.205387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:12.205387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:12.205387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:12.205387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:12.205387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:12.205387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:12.205387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:12.205387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:12.205387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:12.206180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:12.206180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:12.206180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:12.206180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:12.206180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:12.206180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:12.206180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:12.206180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:12.206180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:12.206180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:12.206180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:42:12.206180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:12.208582 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 215/429] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:42:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:12.214183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23980b-d5e5-4a19-b6cc-ec2b4efb6f56 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:12.286847 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85446c-d2ee-4a8d-bd09-e7028bfffc56 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:12.297044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:12.322480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:12.337520 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615b631-4445-4ff3-ac2f-588573d47e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:12.338561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615b631-4445-4ff3-ac2f-588573d47e22 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:12.339110 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615b631-4445-4ff3-ac2f-588573d47e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:12.339637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615b631-4445-4ff3-ac2f-588573d47e22 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:12.340756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615b631-4445-4ff3-ac2f-588573d47e22 None admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:12.341356 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615b631-4445-4ff3-ac2f-588573d47e22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:12.343235 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615b631-4445-4ff3-ac2f-588573d47e22 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:12.350887 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23980b-d5e5-4a19-b6cc-ec2b4efb6f56 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:12.357880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 215/430] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:42:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:12.363513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615b631-4445-4ff3-ac2f-588573d47e22 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:12.406332 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03a5cf-8ab8-4201-833c-b18dea3d46ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:12.410240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03a5cf-8ab8-4201-833c-b18dea3d46ab None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:12.410961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03a5cf-8ab8-4201-833c-b18dea3d46ab None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:12.411462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03a5cf-8ab8-4201-833c-b18dea3d46ab None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:12.412710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03a5cf-8ab8-4201-833c-b18dea3d46ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:12.413505 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03a5cf-8ab8-4201-833c-b18dea3d46ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:12.415868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03a5cf-8ab8-4201-833c-b18dea3d46ab None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:12.429978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 216/431] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:12.437220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03a5cf-8ab8-4201-833c-b18dea3d46ab None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:12.507965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615b631-4445-4ff3-ac2f-588573d47e22 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:12.547919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f4424-386d-4e74-b805-5b879bde265c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:12.549026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f4424-386d-4e74-b805-5b879bde265c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:12.549301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f4424-386d-4e74-b805-5b879bde265c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:12.549571 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f4424-386d-4e74-b805-5b879bde265c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:12.573644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03a5cf-8ab8-4201-833c-b18dea3d46ab None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:12.595155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:12.612589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:12.620589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e214db-6435-4d6b-8585-0267bf766fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:12.621202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e214db-6435-4d6b-8585-0267bf766fae None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:12.621481 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e214db-6435-4d6b-8585-0267bf766fae None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:12.621754 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e214db-6435-4d6b-8585-0267bf766fae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:12.625696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f4424-386d-4e74-b805-5b879bde265c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:12.628142 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f4424-386d-4e74-b805-5b879bde265c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:12.698815 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e214db-6435-4d6b-8585-0267bf766fae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:12.700302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e214db-6435-4d6b-8585-0267bf766fae None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:12.746534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 216/432] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 07:42:12.746534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12701f9b-ffa7-46d4-a8f2-b4bd39f2f601 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:12.746534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12701f9b-ffa7-46d4-a8f2-b4bd39f2f601 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:12.746534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12701f9b-ffa7-46d4-a8f2-b4bd39f2f601 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:12.746534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 217/433] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:12.747515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12701f9b-ffa7-46d4-a8f2-b4bd39f2f601 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:12.815701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 217/434] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:12.817284 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b71ac-a362-4d92-8e7a-654f879898e5 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:12.817781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b71ac-a362-4d92-8e7a-654f879898e5 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:12.818244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b71ac-a362-4d92-8e7a-654f879898e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:12.838370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-069b71ac-a362-4d92-8e7a-654f879898e5 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:12.839544 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-069b71ac-a362-4d92-8e7a-654f879898e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:12.847820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:12.854179 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:12.862080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12701f9b-ffa7-46d4-a8f2-b4bd39f2f601 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:12.895700 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a101f-19e5-4fbe-ab58-6ca995265546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:12.896346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a101f-19e5-4fbe-ab58-6ca995265546 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:12.896615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a101f-19e5-4fbe-ab58-6ca995265546 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:12.896884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a101f-19e5-4fbe-ab58-6ca995265546 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:12.897969 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a101f-19e5-4fbe-ab58-6ca995265546 None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:12.899393 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a101f-19e5-4fbe-ab58-6ca995265546 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:12.901017 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 218/435] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:42:12 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:12.905484 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a101f-19e5-4fbe-ab58-6ca995265546 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:12.906674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 218/436] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:12.908515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844a260-9a84-48fb-96a9-164086412858 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:12.908803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844a260-9a84-48fb-96a9-164086412858 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:12.909074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844a260-9a84-48fb-96a9-164086412858 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:12.910597 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 219/437] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:12.911302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844a260-9a84-48fb-96a9-164086412858 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:13.018003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a101f-19e5-4fbe-ab58-6ca995265546 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:13.018987 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844a260-9a84-48fb-96a9-164086412858 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:13.053938 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e81eb-4215-43d5-ac3c-f25cb419460e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:13.054643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e81eb-4215-43d5-ac3c-f25cb419460e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:13.054970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e81eb-4215-43d5-ac3c-f25cb419460e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:13.055187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e81eb-4215-43d5-ac3c-f25cb419460e None admin] PATH_INFO: `/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:13.056360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5fd88-b3fd-4a0b-a066-dc02cd52cdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:13.056475 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e81eb-4215-43d5-ac3c-f25cb419460e None admin] RBAC: Authorizing `identity:get_project(project_id=4383a0cd1b1e46558d94a55c2ac3f8d5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:13.056991 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5fd88-b3fd-4a0b-a066-dc02cd52cdae None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:13.057266 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5fd88-b3fd-4a0b-a066-dc02cd52cdae None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:13.057534 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5fd88-b3fd-4a0b-a066-dc02cd52cdae None admin] PATH_INFO: `/v3/groups/a06ef62241eb4ae4b5ccbec174139142` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:13.057912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e81eb-4215-43d5-ac3c-f25cb419460e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:13.058689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5fd88-b3fd-4a0b-a066-dc02cd52cdae None admin] RBAC: Authorizing `identity:get_group(group_id=a06ef62241eb4ae4b5ccbec174139142)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:13.060137 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5fd88-b3fd-4a0b-a066-dc02cd52cdae None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:13.066633 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 220/438] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 7 07:42:12 2022] GET /identity/v3/groups/a06ef62241eb4ae4b5ccbec174139142 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:13.066633 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f0bed-4e8a-4515-b79e-e7557211d069 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:13.066633 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f0bed-4e8a-4515-b79e-e7557211d069 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:13.066633 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f0bed-4e8a-4515-b79e-e7557211d069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:13.066915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 219/439] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:42:12 2022] GET /identity/v3/projects/4383a0cd1b1e46558d94a55c2ac3f8d5 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:13.066915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e81eb-4215-43d5-ac3c-f25cb419460e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:13.082001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-423f0bed-4e8a-4515-b79e-e7557211d069 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:13.083191 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-423f0bed-4e8a-4515-b79e-e7557211d069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:13.093706 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:13.094140 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:13.164104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 221/440] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:13.164104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f0bed-4e8a-4515-b79e-e7557211d069 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:13.186412 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e81eb-4215-43d5-ac3c-f25cb419460e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:13.223080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cdd01-132b-4bab-b079-5b9df18b051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:13.223701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cdd01-132b-4bab-b079-5b9df18b051f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:13.223989 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cdd01-132b-4bab-b079-5b9df18b051f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:13.224270 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cdd01-132b-4bab-b079-5b9df18b051f None admin] PATH_INFO: `/v3/groups/a06ef62241eb4ae4b5ccbec174139142` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:13.225482 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86cdd01-132b-4bab-b079-5b9df18b051f None admin] RBAC: Authorizing `identity:get_group(group_id=a06ef62241eb4ae4b5ccbec174139142)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:13.227001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86cdd01-132b-4bab-b079-5b9df18b051f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:13.228763 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 220/441] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 7 07:42:13 2022] GET /identity/v3/groups/a06ef62241eb4ae4b5ccbec174139142 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:13.229380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cdd01-132b-4bab-b079-5b9df18b051f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:13.269747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f0bed-4e8a-4515-b79e-e7557211d069 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:13.303757 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16991fd8-ba85-4165-9537-a71ca6e6f8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:13.304396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16991fd8-ba85-4165-9537-a71ca6e6f8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:13.304673 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16991fd8-ba85-4165-9537-a71ca6e6f8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:13.304940 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16991fd8-ba85-4165-9537-a71ca6e6f8d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:13.305652 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16991fd8-ba85-4165-9537-a71ca6e6f8d7 None admin] RBAC: Adding query filter params (group.id=a06ef62241eb4ae4b5ccbec174139142, role.id=6ebbed85905048a198e4261a3510dafc, scope.project.id=ea579b172f214888a95d9af5a4e1d122) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:13.305954 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16991fd8-ba85-4165-9537-a71ca6e6f8d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:13.307386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16991fd8-ba85-4165-9537-a71ca6e6f8d7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:13.313694 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 222/442] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 7 07:42:13 2022] GET /identity/v3/role_assignments?group.id=a06ef62241eb4ae4b5ccbec174139142&scope.project.id=ea579b172f214888a95d9af5a4e1d122&role.id=6ebbed85905048a198e4261a3510dafc => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:13.314577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16991fd8-ba85-4165-9537-a71ca6e6f8d7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:13.332996 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:13.336903 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:13.347623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cdd01-132b-4bab-b079-5b9df18b051f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:13.391678 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02493ab5-1815-4be1-80ad-4937bf9b7a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:13.393052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02493ab5-1815-4be1-80ad-4937bf9b7a29 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:13.393802 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02493ab5-1815-4be1-80ad-4937bf9b7a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:13.394221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02493ab5-1815-4be1-80ad-4937bf9b7a29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:13.406949 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02493ab5-1815-4be1-80ad-4937bf9b7a29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:13.418613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-02493ab5-1815-4be1-80ad-4937bf9b7a29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:42:13.418613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:13.418613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:13.418613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:13.418613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:13.418613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:13.418613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:13.418613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:13.418613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:13.418613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:13.418613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:13.418613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:13.418613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:13.418613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:13.418613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:13.418613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:13.419718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:13.419718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:13.419718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:13.419718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:13.419718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:13.419718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:13.419718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:13.419718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:13.419718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:13.419718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:13.419718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:13.419718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:13.419718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:13.419718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:13.419718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:13.419718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:13.420632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:13.420632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:13.420632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:13.420632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:13.420632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:13.420632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:13.420632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:13.420632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:13.420632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:13.420632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:13.420632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:42:13.420632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:13.423309 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 221/443] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:42:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:13.432124 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02493ab5-1815-4be1-80ad-4937bf9b7a29 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:13.458973 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16991fd8-ba85-4165-9537-a71ca6e6f8d7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:13.500671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bc750-a730-4deb-97f3-9cdc76d3a3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:13.502287 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bc750-a730-4deb-97f3-9cdc76d3a3ce None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:13.502574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bc750-a730-4deb-97f3-9cdc76d3a3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:13.502863 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bc750-a730-4deb-97f3-9cdc76d3a3ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:13.503542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bc750-a730-4deb-97f3-9cdc76d3a3ce None admin] RBAC: Adding query filter params (group.id=a06ef62241eb4ae4b5ccbec174139142, role.id=84a30faea3e94eb48b03ec7ceb7c1392, scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:13.503849 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bc750-a730-4deb-97f3-9cdc76d3a3ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:13.505302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bc750-a730-4deb-97f3-9cdc76d3a3ce None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:13.512023 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 223/444] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 7 07:42:13 2022] GET /identity/v3/role_assignments?group.id=a06ef62241eb4ae4b5ccbec174139142&scope.project.id=4383a0cd1b1e46558d94a55c2ac3f8d5&role.id=84a30faea3e94eb48b03ec7ceb7c1392 => generated 611 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:13.512849 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bc750-a730-4deb-97f3-9cdc76d3a3ce None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:13.553565 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02493ab5-1815-4be1-80ad-4937bf9b7a29 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:13.599900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d019014-b8d1-4391-849a-cd714be4ed8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:13.600487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d019014-b8d1-4391-849a-cd714be4ed8f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:13.600764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d019014-b8d1-4391-849a-cd714be4ed8f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:13.601035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d019014-b8d1-4391-849a-cd714be4ed8f None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:13.601707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d019014-b8d1-4391-849a-cd714be4ed8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:13.602008 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d019014-b8d1-4391-849a-cd714be4ed8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:13.603418 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d019014-b8d1-4391-849a-cd714be4ed8f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:13.612046 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:13.613368 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 222/445] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:13.619316 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d019014-b8d1-4391-849a-cd714be4ed8f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:13.638859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:13.641690 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bc750-a730-4deb-97f3-9cdc76d3a3ce None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:13.682063 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c7bf4-ebb9-4004-adeb-c0df7d11f5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:13.682799 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c7bf4-ebb9-4004-adeb-c0df7d11f5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:13.683131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c7bf4-ebb9-4004-adeb-c0df7d11f5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:13.683471 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c7bf4-ebb9-4004-adeb-c0df7d11f5a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:13.697844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c7bf4-ebb9-4004-adeb-c0df7d11f5a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:13.702618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-688c7bf4-ebb9-4004-adeb-c0df7d11f5a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:42:13.702618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:13.702618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:13.702618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:13.702618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:13.702618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:13.702618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:13.702618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:13.702618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:13.702618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:13.702618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:13.702618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:13.702618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:13.702618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:13.702618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:13.702618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:13.703499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:13.703499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:13.703499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:13.703499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:13.703499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:13.703499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:13.703499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:13.703499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:13.703499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:13.703499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:13.703499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:13.703499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:13.703499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:13.703499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:13.703499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:13.703499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:13.704879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:13.704879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:13.704879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:13.704879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:13.704879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:13.704879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:13.704879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:13.704879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:13.704879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:13.704879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:13.704879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:42:13.704879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:13.704879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 224/446] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:42:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:13.709974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c7bf4-ebb9-4004-adeb-c0df7d11f5a5 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:13.742600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d019014-b8d1-4391-849a-cd714be4ed8f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:13.785969 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335edd7a-db18-4a5b-9789-7c8d9a6892ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:13.787145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335edd7a-db18-4a5b-9789-7c8d9a6892ad None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:13.787359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335edd7a-db18-4a5b-9789-7c8d9a6892ad None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:13.787639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335edd7a-db18-4a5b-9789-7c8d9a6892ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:13.823734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c7bf4-ebb9-4004-adeb-c0df7d11f5a5 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:13.862911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0788b6-23c5-44ca-8da4-9e53ba9d8489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:13.863416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0788b6-23c5-44ca-8da4-9e53ba9d8489 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:13.863694 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0788b6-23c5-44ca-8da4-9e53ba9d8489 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:13.863987 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0788b6-23c5-44ca-8da4-9e53ba9d8489 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:13.864652 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0788b6-23c5-44ca-8da4-9e53ba9d8489 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:13.864951 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0788b6-23c5-44ca-8da4-9e53ba9d8489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:13.866329 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0788b6-23c5-44ca-8da4-9e53ba9d8489 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:13.867791 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:13.874023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335edd7a-db18-4a5b-9789-7c8d9a6892ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:13.875472 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335edd7a-db18-4a5b-9789-7c8d9a6892ad None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:13.878212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 225/447] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:13.883800 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0788b6-23c5-44ca-8da4-9e53ba9d8489 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:13.896172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:14.003344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 223/448] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 07:42:14.008422 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0788b6-23c5-44ca-8da4-9e53ba9d8489 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:14.010740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335edd7a-db18-4a5b-9789-7c8d9a6892ad None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:14.050439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb126d8-6210-459c-b3e9-d6bb80051699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:14.051044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb126d8-6210-459c-b3e9-d6bb80051699 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:14.051321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb126d8-6210-459c-b3e9-d6bb80051699 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:14.051591 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb126d8-6210-459c-b3e9-d6bb80051699 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:14.114755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:14.115324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335edd7a-db18-4a5b-9789-7c8d9a6892ad None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:14.121216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb126d8-6210-459c-b3e9-d6bb80051699 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:14.122667 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb126d8-6210-459c-b3e9-d6bb80051699 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:14.130495 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:14.150546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49719c-8bcd-4411-8528-e50f50d4fc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:14.151166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49719c-8bcd-4411-8528-e50f50d4fc64 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:14.151444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49719c-8bcd-4411-8528-e50f50d4fc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:14.151717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49719c-8bcd-4411-8528-e50f50d4fc64 None admin] PATH_INFO: `/v3/projects/bb18598c40b6447faef02991e1b9c4ed` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:14.152867 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49719c-8bcd-4411-8528-e50f50d4fc64 None admin] RBAC: Authorizing `identity:get_project(project_id=bb18598c40b6447faef02991e1b9c4ed)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:14.154274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49719c-8bcd-4411-8528-e50f50d4fc64 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:14.155877 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 224/449] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:42:14 2022] GET /identity/v3/projects/bb18598c40b6447faef02991e1b9c4ed => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:14.161492 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49719c-8bcd-4411-8528-e50f50d4fc64 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:14.224709 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 226/450] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 07:42:14.230392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb126d8-6210-459c-b3e9-d6bb80051699 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:14.258044 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49719c-8bcd-4411-8528-e50f50d4fc64 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:14.296460 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fde12-f453-46d4-9aa0-18fc14e36f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:14.297075 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fde12-f453-46d4-9aa0-18fc14e36f32 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:14.297344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fde12-f453-46d4-9aa0-18fc14e36f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:14.297614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fde12-f453-46d4-9aa0-18fc14e36f32 None admin] PATH_INFO: `/v3/groups/1782e6af28b34561884d3ba44a55628b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:14.298881 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45fde12-f453-46d4-9aa0-18fc14e36f32 None admin] RBAC: Authorizing `identity:get_group(group_id=1782e6af28b34561884d3ba44a55628b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:14.300323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45fde12-f453-46d4-9aa0-18fc14e36f32 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:14.301981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 225/451] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 7 07:42:14 2022] GET /identity/v3/groups/1782e6af28b34561884d3ba44a55628b => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:14.307656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fde12-f453-46d4-9aa0-18fc14e36f32 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:14.325770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:14.332396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb126d8-6210-459c-b3e9-d6bb80051699 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:14.356789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:14.366235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f310e-fab4-44d8-9ead-1352a15ee513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:14.366796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f310e-fab4-44d8-9ead-1352a15ee513 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:14.367068 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f310e-fab4-44d8-9ead-1352a15ee513 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:14.367336 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f310e-fab4-44d8-9ead-1352a15ee513 None admin] PATH_INFO: `/v3/projects/ea579b172f214888a95d9af5a4e1d122` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:14.368461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f310e-fab4-44d8-9ead-1352a15ee513 None admin] RBAC: Authorizing `identity:get_project(project_id=ea579b172f214888a95d9af5a4e1d122)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:14.369908 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f310e-fab4-44d8-9ead-1352a15ee513 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:14.371506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 227/452] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:42:14 2022] GET /identity/v3/projects/ea579b172f214888a95d9af5a4e1d122 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:14.377114 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f310e-fab4-44d8-9ead-1352a15ee513 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:14.406606 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fde12-f453-46d4-9aa0-18fc14e36f32 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:14.442485 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccfbd0-a02e-47c2-80e0-9df6486b9b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:14.443076 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccfbd0-a02e-47c2-80e0-9df6486b9b0c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:14.443351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccfbd0-a02e-47c2-80e0-9df6486b9b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:14.443617 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccfbd0-a02e-47c2-80e0-9df6486b9b0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:14.444618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccfbd0-a02e-47c2-80e0-9df6486b9b0c None admin] RBAC: Adding query filter params (group.id=1782e6af28b34561884d3ba44a55628b, role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=bb18598c40b6447faef02991e1b9c4ed) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:14.444920 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccfbd0-a02e-47c2-80e0-9df6486b9b0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:14.446337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccfbd0-a02e-47c2-80e0-9df6486b9b0c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:14.451946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 226/453] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 7 07:42:14 2022] GET /identity/v3/role_assignments?group.id=1782e6af28b34561884d3ba44a55628b&scope.project.id=bb18598c40b6447faef02991e1b9c4ed&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:14.483557 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f310e-fab4-44d8-9ead-1352a15ee513 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:14.519970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6393eb-b310-4cac-9f7a-fca4a85d386e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:14.520636 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6393eb-b310-4cac-9f7a-fca4a85d386e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:14.520910 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6393eb-b310-4cac-9f7a-fca4a85d386e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:14.521173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6393eb-b310-4cac-9f7a-fca4a85d386e None admin] PATH_INFO: `/v3/groups/a06ef62241eb4ae4b5ccbec174139142` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:14.522318 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6393eb-b310-4cac-9f7a-fca4a85d386e None admin] RBAC: Authorizing `identity:get_group(group_id=a06ef62241eb4ae4b5ccbec174139142)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:14.523785 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6393eb-b310-4cac-9f7a-fca4a85d386e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:14.525617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 228/454] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 7 07:42:14 2022] GET /identity/v3/groups/a06ef62241eb4ae4b5ccbec174139142 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:14.531442 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccfbd0-a02e-47c2-80e0-9df6486b9b0c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:14.637503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccfbd0-a02e-47c2-80e0-9df6486b9b0c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:14.651610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:14.672627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c02ce-8222-4fc5-aefa-77bb2ea9b2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:14.673508 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c02ce-8222-4fc5-aefa-77bb2ea9b2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:14.673778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c02ce-8222-4fc5-aefa-77bb2ea9b2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:14.674049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c02ce-8222-4fc5-aefa-77bb2ea9b2c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:14.674769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c02ce-8222-4fc5-aefa-77bb2ea9b2c6 None admin] RBAC: Adding query filter params (group.id=a06ef62241eb4ae4b5ccbec174139142, role.id=84a30faea3e94eb48b03ec7ceb7c1392, scope.project.id=ea579b172f214888a95d9af5a4e1d122) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:14.675075 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c02ce-8222-4fc5-aefa-77bb2ea9b2c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:14.676557 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c02ce-8222-4fc5-aefa-77bb2ea9b2c6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:14.682289 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 227/455] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 7 07:42:14 2022] GET /identity/v3/role_assignments?group.id=a06ef62241eb4ae4b5ccbec174139142&scope.project.id=ea579b172f214888a95d9af5a4e1d122&role.id=84a30faea3e94eb48b03ec7ceb7c1392 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:17.513406 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f2e97-94f1-4873-89c1-fa700ae0f000 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:17.513976 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f2e97-94f1-4873-89c1-fa700ae0f000 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:17.513976 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f2e97-94f1-4873-89c1-fa700ae0f000 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:17.515648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 229/456] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:17.521558 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee3be6-3f24-4699-a6b2-20414a117281 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:17.522049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee3be6-3f24-4699-a6b2-20414a117281 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:17.522140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee3be6-3f24-4699-a6b2-20414a117281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:17.561149 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e8ee3be6-3f24-4699-a6b2-20414a117281 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:17.562264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e8ee3be6-3f24-4699-a6b2-20414a117281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:17.595871 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d899ad7-e681-4d96-af3c-9db565c6867c None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:17.596204 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d899ad7-e681-4d96-af3c-9db565c6867c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:17.596441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d899ad7-e681-4d96-af3c-9db565c6867c None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:17.601846 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 230/457] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:42:17.603616 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b21164-43f2-4ee2-b78a-0a8553d09da0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:17.603901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b21164-43f2-4ee2-b78a-0a8553d09da0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:17.604202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b21164-43f2-4ee2-b78a-0a8553d09da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:17.646090 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-78b21164-43f2-4ee2-b78a-0a8553d09da0 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:17.647181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-78b21164-43f2-4ee2-b78a-0a8553d09da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:17.648863 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 228/458] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:42:17.675101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:17.739019 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 231/459] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:42:18.005137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee3be6-3f24-4699-a6b2-20414a117281 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:18.020892 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fbdf0-e2ee-4824-a132-8feb49dfa300 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:18.021183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fbdf0-e2ee-4824-a132-8feb49dfa300 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:18.021434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fbdf0-e2ee-4824-a132-8feb49dfa300 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:18.023020 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 232/460] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:18.028059 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb578a-3b3d-48a5-b3d2-cb91c55da715 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:18.028594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb578a-3b3d-48a5-b3d2-cb91c55da715 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:18.029148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb578a-3b3d-48a5-b3d2-cb91c55da715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:18.049335 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e9cb578a-3b3d-48a5-b3d2-cb91c55da715 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:18.050915 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e9cb578a-3b3d-48a5-b3d2-cb91c55da715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:18.162960 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 233/461] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:42:18.170296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee3be6-3f24-4699-a6b2-20414a117281 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:18.178674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb578a-3b3d-48a5-b3d2-cb91c55da715 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:18.186316 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:18.252848 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e891510-2472-465e-bc8a-0a0acffd3baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:18.254642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e891510-2472-465e-bc8a-0a0acffd3baa None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:18.255357 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e891510-2472-465e-bc8a-0a0acffd3baa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:18.257076 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e891510-2472-465e-bc8a-0a0acffd3baa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:18.279046 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e891510-2472-465e-bc8a-0a0acffd3baa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:18.285660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e891510-2472-465e-bc8a-0a0acffd3baa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:18.311454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-8e891510-2472-465e-bc8a-0a0acffd3baa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:18.311454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:18.311454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:18.311454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:18.311454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:18.311454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:18.311454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:18.311454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:18.311454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:18.311454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:18.311454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:18.311454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:18.311454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:18.311454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:18.311454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:18.311454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:18.312452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:18.313300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:18.313300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:18.313300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:18.314750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:18.319808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 229/462] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:18.325859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb578a-3b3d-48a5-b3d2-cb91c55da715 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:18.326954 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e891510-2472-465e-bc8a-0a0acffd3baa None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:18.399640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777f7e9-e9f0-4967-8455-981fae290444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:18.400310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777f7e9-e9f0-4967-8455-981fae290444 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:18.400590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777f7e9-e9f0-4967-8455-981fae290444 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:18.400859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777f7e9-e9f0-4967-8455-981fae290444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:18.417374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777f7e9-e9f0-4967-8455-981fae290444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:18.418918 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777f7e9-e9f0-4967-8455-981fae290444 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:18.436701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-f777f7e9-e9f0-4967-8455-981fae290444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:18.436701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:18.436701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:18.436701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:18.436701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:18.436701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:18.436701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:18.436701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:18.436701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:18.436701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:18.436701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:18.436701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:18.436701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:18.436701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:18.436701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:18.436701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:18.437717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:18.438545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:18.438545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:18.438545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:18.438545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 234/463] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:18.440716 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d223a2-a380-4c25-a847-a0b60a93aefe None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:18.440716 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d223a2-a380-4c25-a847-a0b60a93aefe None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:18.440905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d223a2-a380-4c25-a847-a0b60a93aefe None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:18.446735 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 235/464] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:18.447562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d223a2-a380-4c25-a847-a0b60a93aefe None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:18.474808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e891510-2472-465e-bc8a-0a0acffd3baa None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:18.513167 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422d83a-a56a-4e9e-993b-31bb2504c754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:18.514216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422d83a-a56a-4e9e-993b-31bb2504c754 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:18.514736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422d83a-a56a-4e9e-993b-31bb2504c754 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:18.515181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422d83a-a56a-4e9e-993b-31bb2504c754 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:18.516022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422d83a-a56a-4e9e-993b-31bb2504c754 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:18.516490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422d83a-a56a-4e9e-993b-31bb2504c754 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:18.518400 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422d83a-a56a-4e9e-993b-31bb2504c754 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:18.532284 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 230/465] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:18.533676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8f18b-3a35-4524-a73e-127b47122260 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:18.534034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8f18b-3a35-4524-a73e-127b47122260 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:18.534290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8f18b-3a35-4524-a73e-127b47122260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:18.552058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-92c8f18b-3a35-4524-a73e-127b47122260 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:18.553252 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-92c8f18b-3a35-4524-a73e-127b47122260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:18.563338 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d223a2-a380-4c25-a847-a0b60a93aefe None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:18.570062 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:18.604117 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314444ed-9281-47f3-a6a7-517708039a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:18.604622 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314444ed-9281-47f3-a6a7-517708039a23 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:18.604897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314444ed-9281-47f3-a6a7-517708039a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:18.605212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314444ed-9281-47f3-a6a7-517708039a23 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:18.605781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314444ed-9281-47f3-a6a7-517708039a23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:18.606077 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314444ed-9281-47f3-a6a7-517708039a23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:18.607463 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314444ed-9281-47f3-a6a7-517708039a23 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:18.620570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 236/466] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:18.621258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314444ed-9281-47f3-a6a7-517708039a23 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:18.625405 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:18.634573 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 231/467] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:18.637704 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8f18b-3a35-4524-a73e-127b47122260 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:18.736843 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314444ed-9281-47f3-a6a7-517708039a23 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:18.775299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8f18b-3a35-4524-a73e-127b47122260 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:18.789471 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d094fb4-c9af-446a-bb8c-216821ce187c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:18.790409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d094fb4-c9af-446a-bb8c-216821ce187c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:18.790884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d094fb4-c9af-446a-bb8c-216821ce187c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:18.791383 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d094fb4-c9af-446a-bb8c-216821ce187c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:18.806195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d094fb4-c9af-446a-bb8c-216821ce187c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:18.809014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d094fb4-c9af-446a-bb8c-216821ce187c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:18.812469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b27201-f99f-4587-89c6-e97476b06f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:18.813012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b27201-f99f-4587-89c6-e97476b06f8e None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:18.813394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b27201-f99f-4587-89c6-e97476b06f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:18.814348 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b27201-f99f-4587-89c6-e97476b06f8e None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:18.815292 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b27201-f99f-4587-89c6-e97476b06f8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:18.816859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b27201-f99f-4587-89c6-e97476b06f8e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:18.829039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-2d094fb4-c9af-446a-bb8c-216821ce187c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:18.829039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:18.829039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:18.829039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:18.829039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:18.829039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:18.829039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:18.829039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:18.829039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:18.829039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:18.829039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:18.829039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:18.829039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:18.829039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:18.829039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:18.829039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:18.830298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:18.831418 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:18.831418 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:18.831418 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:18.833391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 237/468] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:18.834470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d094fb4-c9af-446a-bb8c-216821ce187c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:18.848923 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 232/469] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:42:18 2022] POST /identity/v3/users => generated 246 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:42:18.850099 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b27201-f99f-4587-89c6-e97476b06f8e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:18.860233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:18.921451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:18.977045 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d094fb4-c9af-446a-bb8c-216821ce187c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:18.980301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b27201-f99f-4587-89c6-e97476b06f8e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:19.014824 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ee3e2-4b31-453b-a3d8-09917f3b9fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:19.015435 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ee3e2-4b31-453b-a3d8-09917f3b9fa4 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:19.015710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ee3e2-4b31-453b-a3d8-09917f3b9fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:19.015974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ee3e2-4b31-453b-a3d8-09917f3b9fa4 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:19.016985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ee3e2-4b31-453b-a3d8-09917f3b9fa4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:19.018444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ee3e2-4b31-453b-a3d8-09917f3b9fa4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:19.018686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1ff4a-778a-497a-8087-8ae7cb0fc497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:19.019295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1ff4a-778a-497a-8087-8ae7cb0fc497 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:19.019569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1ff4a-778a-497a-8087-8ae7cb0fc497 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:19.019836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1ff4a-778a-497a-8087-8ae7cb0fc497 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:19.020507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d1ff4a-778a-497a-8087-8ae7cb0fc497 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:19.020805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d1ff4a-778a-497a-8087-8ae7cb0fc497 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:19.022145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d1ff4a-778a-497a-8087-8ae7cb0fc497 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:19.040242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 233/470] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:19.041592 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e94f9-89e9-4021-8c3a-4c6d79e46097 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:19.041893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e94f9-89e9-4021-8c3a-4c6d79e46097 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:19.042162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e94f9-89e9-4021-8c3a-4c6d79e46097 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:19.043412 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 234/471] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:19.043664 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 238/472] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:42:18 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:42:19.044160 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e94f9-89e9-4021-8c3a-4c6d79e46097 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:19.044490 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ee3e2-4b31-453b-a3d8-09917f3b9fa4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:19.142872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:19.178695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ee3e2-4b31-453b-a3d8-09917f3b9fa4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:19.193918 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:19.197156 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e94f9-89e9-4021-8c3a-4c6d79e46097 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:19.227511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3a58a-0285-4446-94ea-b11312476a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:19.228291 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3a58a-0285-4446-94ea-b11312476a3f None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:19.228578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3a58a-0285-4446-94ea-b11312476a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:19.228849 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3a58a-0285-4446-94ea-b11312476a3f None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:19.229661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3a58a-0285-4446-94ea-b11312476a3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:19.231821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3a58a-0285-4446-94ea-b11312476a3f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:19.240152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e97c04-ecb4-4e95-9084-344a0ba1a44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:19.240780 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e97c04-ecb4-4e95-9084-344a0ba1a44b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:19.241056 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e97c04-ecb4-4e95-9084-344a0ba1a44b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:19.241322 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e97c04-ecb4-4e95-9084-344a0ba1a44b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:19.257826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e97c04-ecb4-4e95-9084-344a0ba1a44b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:19.258830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e97c04-ecb4-4e95-9084-344a0ba1a44b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:19.259884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 239/473] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:42:19 2022] POST /identity/v3/users => generated 248 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:42:19.261512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f123ae-8513-4464-92fc-2a1e4d24c168 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:19.261796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f123ae-8513-4464-92fc-2a1e4d24c168 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:19.262276 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f123ae-8513-4464-92fc-2a1e4d24c168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:19.282212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-30e97c04-ecb4-4e95-9084-344a0ba1a44b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:19.282212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:19.282212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:19.282212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:19.282212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:19.282212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:19.282212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:19.282212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:19.282212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:19.282212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:19.282212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:19.282212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:19.282212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:19.282212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:19.282212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:19.282212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:19.283759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:19.284644 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:19.284644 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:19.284644 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:19.284644 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 235/474] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:19.292233 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-98f123ae-8513-4464-92fc-2a1e4d24c168 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:19.292394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e97c04-ecb4-4e95-9084-344a0ba1a44b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:19.293456 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-98f123ae-8513-4464-92fc-2a1e4d24c168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:19.392423 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 240/475] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:19.432271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e97c04-ecb4-4e95-9084-344a0ba1a44b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:19.452577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:19.472099 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a932f9e-131f-4c58-add9-f285362d77c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:19.472725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a932f9e-131f-4c58-add9-f285362d77c0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:19.473079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a932f9e-131f-4c58-add9-f285362d77c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:19.473611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a932f9e-131f-4c58-add9-f285362d77c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:19.475282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a932f9e-131f-4c58-add9-f285362d77c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:19.475282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a932f9e-131f-4c58-add9-f285362d77c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:19.476187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a932f9e-131f-4c58-add9-f285362d77c0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:19.489535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 236/476] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:19.494639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f123ae-8513-4464-92fc-2a1e4d24c168 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:19.599713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:19.604405 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f123ae-8513-4464-92fc-2a1e4d24c168 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:19.640829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de632a-b6cd-4830-aa28-82faac8868a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:19.641810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de632a-b6cd-4830-aa28-82faac8868a2 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:19.642319 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de632a-b6cd-4830-aa28-82faac8868a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:19.642848 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de632a-b6cd-4830-aa28-82faac8868a2 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:19.643751 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de632a-b6cd-4830-aa28-82faac8868a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:19.645396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de632a-b6cd-4830-aa28-82faac8868a2 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:19.674672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 241/477] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:42:19 2022] POST /identity/v3/users => generated 249 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:42:19.739665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a932f9e-131f-4c58-add9-f285362d77c0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:19.865961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a932f9e-131f-4c58-add9-f285362d77c0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:19.910697 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce600878-26bb-4c5d-b704-35aa84bd1a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:19.911311 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce600878-26bb-4c5d-b704-35aa84bd1a78 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:19.911591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce600878-26bb-4c5d-b704-35aa84bd1a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:19.911862 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce600878-26bb-4c5d-b704-35aa84bd1a78 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:19.912575 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce600878-26bb-4c5d-b704-35aa84bd1a78 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:19.913949 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce600878-26bb-4c5d-b704-35aa84bd1a78 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:19.927181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.sql.core [None req-ce600878-26bb-4c5d-b704-35aa84bd1a78 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 07:42:19.927181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 07 07:42:19.927181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [parameters: {'id': '0480c9f6875345af888b434db191100c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 07 07:42:19.927181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81475) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 07:42:19.944888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ce600878-26bb-4c5d-b704-35aa84bd1a78 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 07 07:42:19.944888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:19.944888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 07:42:19.944888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 07:42:19.944888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 07:42:19.944888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 07:42:19.944888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 07:42:19.944888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 07:42:19.944888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 07:42:19.944888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application conn.query(q) Sep 07 07:42:19.944888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 07:42:19.944888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 07:42:19.944888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 07:42:19.944888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result.read() Sep 07 07:42:19.944888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return ref.to_dict() Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 07:42:19.945903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application next(self.gen) Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application yield resource Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application next(self.gen) Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise self.value Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application session.commit() Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 07:42:19.946872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.session.flush() Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application compat.raise_( Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 07:42:19.948556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 07:42:19.949426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 07:42:19.949426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 07:42:19.949426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 07 07:42:19.949426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 07 07:42:19.949426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 07:42:19.949426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 07:42:19.949426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 07:42:19.949426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 07:42:19.949426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 07:42:19.949426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 07:42:19.949426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 07:42:19.949426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 07:42:19.949426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 07:42:19.949426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 07:42:19.949426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 07:42:19.949426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception Sep 07 07:42:19.950220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 07:42:19.950220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 07:42:19.950220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 07:42:19.950220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 07:42:19.950220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 07:42:19.950220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 07:42:19.950220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 07:42:19.950220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application conn.query(q) Sep 07 07:42:19.950220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 07:42:19.950220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 07:42:19.950220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 07:42:19.950220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result.read() Sep 07 07:42:19.950220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 07:42:19.950220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 07:42:19.950220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 07:42:19.950220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 07:42:19.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 07:42:19.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 07:42:19.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 07:42:19.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 07:42:19.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 07:42:19.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 07 07:42:19.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [parameters: {'id': '0480c9f6875345af888b434db191100c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 07 07:42:19.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 07:42:19.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:19.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:42:19.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:19.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:19.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:19.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:19.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:19.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:19.955622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:19.955622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:19.955622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:19.955622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:19.955622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:19.955622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:19.955622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 07 07:42:19.955622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 07 07:42:19.955622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:19.955622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:19.955622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 07 07:42:19.955622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 07 07:42:19.955622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 07:42:19.955622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 07:42:19.955622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 07 07:42:19.955622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:19.959195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 237/478] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Sep 7 07:42:19 2022] POST /identity/v3/roles => generated 144 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:19.959332 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de632a-b6cd-4830-aa28-82faac8868a2 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:20.091056 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de632a-b6cd-4830-aa28-82faac8868a2 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:20.130733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff7473-2bdb-4524-b310-d544d4cd7a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:20.131730 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff7473-2bdb-4524-b310-d544d4cd7a87 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:20.132240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff7473-2bdb-4524-b310-d544d4cd7a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:20.132761 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff7473-2bdb-4524-b310-d544d4cd7a87 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:20.144458 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff7473-2bdb-4524-b310-d544d4cd7a87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:20.150064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-12ff7473-2bdb-4524-b310-d544d4cd7a87 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:42:20.150064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:20.150064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:20.150064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:20.150064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:20.150064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:20.150064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:20.150064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:20.150064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:20.150064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:20.150064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:20.150064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:20.150064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:20.150064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:20.150064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:20.150064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:20.150911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:20.150911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:20.150911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:20.150911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:20.150911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:20.150911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:20.150911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:20.150911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:20.150911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:20.150911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:20.150911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:20.150911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:20.150911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:20.150911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:20.150911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:20.150911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:20.151715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:20.151715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:20.151715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:20.151715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:20.151715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:20.151715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:20.151715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:20.151715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:20.151715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:20.151715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:20.151715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 07:42:20.151715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:20.154019 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 242/479] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:42:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:20.160779 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce600878-26bb-4c5d-b704-35aa84bd1a78 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:20.237961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:20.313477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce600878-26bb-4c5d-b704-35aa84bd1a78 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:20.366260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9baaa7-ca34-4ab0-98a6-d5e6447276d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:20.366868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9baaa7-ca34-4ab0-98a6-d5e6447276d9 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:20.367146 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9baaa7-ca34-4ab0-98a6-d5e6447276d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:20.367419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9baaa7-ca34-4ab0-98a6-d5e6447276d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:20.368108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9baaa7-ca34-4ab0-98a6-d5e6447276d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:20.368406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9baaa7-ca34-4ab0-98a6-d5e6447276d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:20.369771 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9baaa7-ca34-4ab0-98a6-d5e6447276d9 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:20.380249 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 238/480] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:21.492269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf330f6-1990-4921-8276-0aa9e730505e None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:21.492808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf330f6-1990-4921-8276-0aa9e730505e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:21.492808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf330f6-1990-4921-8276-0aa9e730505e None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:21.494418 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 243/481] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:21.499835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f60f2-a8a4-4bc3-8c34-f51d53ccce73 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:21.500257 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f60f2-a8a4-4bc3-8c34-f51d53ccce73 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:21.500415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f60f2-a8a4-4bc3-8c34-f51d53ccce73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:21.519206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d61f60f2-a8a4-4bc3-8c34-f51d53ccce73 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:21.520694 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d61f60f2-a8a4-4bc3-8c34-f51d53ccce73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:21.611032 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 239/482] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:21.904515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81cfe0-525c-494f-8fcb-4d3dcf55d99e None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:21.904901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81cfe0-525c-494f-8fcb-4d3dcf55d99e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:21.905093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81cfe0-525c-494f-8fcb-4d3dcf55d99e None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:21.906620 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 244/483] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:21.912521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f39f2-84a7-42e6-8c68-1566287e762f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:21.912805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f39f2-84a7-42e6-8c68-1566287e762f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:21.913077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f39f2-84a7-42e6-8c68-1566287e762f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:21.939140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9a3f39f2-84a7-42e6-8c68-1566287e762f None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:21.940341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9a3f39f2-84a7-42e6-8c68-1566287e762f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:21.979173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:22.021461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81cfe0-525c-494f-8fcb-4d3dcf55d99e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:22.037860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 240/484] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:22.052228 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:22.178157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81cfe0-525c-494f-8fcb-4d3dcf55d99e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:22.225403 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498a0d0-763c-4241-af76-8307abeb4988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:22.226038 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498a0d0-763c-4241-af76-8307abeb4988 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:22.226316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498a0d0-763c-4241-af76-8307abeb4988 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:22.226590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498a0d0-763c-4241-af76-8307abeb4988 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:22.239264 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498a0d0-763c-4241-af76-8307abeb4988 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:22.244200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-0498a0d0-763c-4241-af76-8307abeb4988 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:22.244200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:22.244200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:22.244200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:22.244200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:22.244200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:22.244200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:22.244200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:22.244200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:22.244200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:22.244200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:22.244200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:22.244200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:22.244200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:22.244200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:22.244200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:22.245244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:22.245244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:22.245244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:22.245244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:22.245244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:22.245244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:22.245244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:22.245244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:22.245244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:22.245244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:22.245244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:22.245244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:22.245244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:22.245244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:22.245244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:22.245244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:22.246360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:22.246360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:22.246360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:22.246360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:22.246360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:22.246360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:22.246360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:22.246360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:22.246360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:22.246360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:22.246360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:22.246360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:22.246360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 245/485] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:22.252726 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f39f2-84a7-42e6-8c68-1566287e762f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:22.289227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493c5c6-3b8a-48c2-bb7d-71536fb874c2 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:22.289506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493c5c6-3b8a-48c2-bb7d-71536fb874c2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:22.289787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493c5c6-3b8a-48c2-bb7d-71536fb874c2 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:22.291475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 246/486] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:22.297048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d03741-9c69-4950-995c-26c987b12eb9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:22.297350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d03741-9c69-4950-995c-26c987b12eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:22.297735 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d03741-9c69-4950-995c-26c987b12eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:22.330733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-48d03741-9c69-4950-995c-26c987b12eb9 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:22.333788 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-48d03741-9c69-4950-995c-26c987b12eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:22.426457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f39f2-84a7-42e6-8c68-1566287e762f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:22.447479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 247/487] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:42:22.448962 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d03741-9c69-4950-995c-26c987b12eb9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:22.453005 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:22.467410 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76c079-9d3f-4566-8b69-da864f7593cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:22.468186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76c079-9d3f-4566-8b69-da864f7593cc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:22.468582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76c079-9d3f-4566-8b69-da864f7593cc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:22.468951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76c079-9d3f-4566-8b69-da864f7593cc None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:22.469710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76c079-9d3f-4566-8b69-da864f7593cc None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:22.470929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76c079-9d3f-4566-8b69-da864f7593cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:22.472516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76c079-9d3f-4566-8b69-da864f7593cc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:22.481963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 241/488] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:22.483293 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98dc09-7498-491a-a896-272063083ebe None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:22.483826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98dc09-7498-491a-a896-272063083ebe None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:22.484224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98dc09-7498-491a-a896-272063083ebe None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:22.485723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 242/489] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:22.487138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98dc09-7498-491a-a896-272063083ebe None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:22.520351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:22.588766 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d03741-9c69-4950-995c-26c987b12eb9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:22.616425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98dc09-7498-491a-a896-272063083ebe None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:22.626984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756ef35-9b72-4ee7-b393-08438d5f1847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:22.627718 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756ef35-9b72-4ee7-b393-08438d5f1847 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:22.628106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756ef35-9b72-4ee7-b393-08438d5f1847 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:22.628478 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756ef35-9b72-4ee7-b393-08438d5f1847 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:22.639645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756ef35-9b72-4ee7-b393-08438d5f1847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:22.645577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-3756ef35-9b72-4ee7-b393-08438d5f1847 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:22.645577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:22.645577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:22.645577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:22.645577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:22.645577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:22.645577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:22.645577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:22.645577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:22.645577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:22.645577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:22.645577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:22.645577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:22.645577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:22.645577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:22.645577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:22.646649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:22.646649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:22.646649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:22.646649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:22.646649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:22.646649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:22.646649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:22.646649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:22.646649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:22.646649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:22.646649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:22.646649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:22.646649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:22.646649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:22.646649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:22.646649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:22.647476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:22.647476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:22.647476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:22.647476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:22.647476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:22.647476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:22.647476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:22.647476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:22.647476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:22.647476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:22.647476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:22.647476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:22.648068 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 248/490] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:22.649903 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b83b1-d88e-4038-86b3-d4b1d8a80446 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:22.650269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b83b1-d88e-4038-86b3-d4b1d8a80446 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:22.650573 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b83b1-d88e-4038-86b3-d4b1d8a80446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:22.662172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09e730-d5d1-4eb5-85aa-ee8a922ad034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:22.663035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09e730-d5d1-4eb5-85aa-ee8a922ad034 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:22.663485 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09e730-d5d1-4eb5-85aa-ee8a922ad034 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:22.663925 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09e730-d5d1-4eb5-85aa-ee8a922ad034 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:22.669883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-868b83b1-d88e-4038-86b3-d4b1d8a80446 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:22.671420 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-868b83b1-d88e-4038-86b3-d4b1d8a80446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:22.678958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09e730-d5d1-4eb5-85aa-ee8a922ad034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:22.681126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09e730-d5d1-4eb5-85aa-ee8a922ad034 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:22.698378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-da09e730-d5d1-4eb5-85aa-ee8a922ad034 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:22.698378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:22.698378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:22.698378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:22.698378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:22.698378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:22.698378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:22.698378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:22.698378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:22.698378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:22.698378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:22.698378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:22.698378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:22.698378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:22.698378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:22.698378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:22.699282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:22.700145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:22.700145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:22.700145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:22.704919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 243/491] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:22.705836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09e730-d5d1-4eb5-85aa-ee8a922ad034 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:22.746920 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:22.756357 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 249/492] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:22.757414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b83b1-d88e-4038-86b3-d4b1d8a80446 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:22.807905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:22.841360 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09e730-d5d1-4eb5-85aa-ee8a922ad034 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:22.878452 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b83b1-d88e-4038-86b3-d4b1d8a80446 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:22.883868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fba51-90b2-4134-a135-bb1a89c07efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:22.884871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fba51-90b2-4134-a135-bb1a89c07efc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:22.885392 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fba51-90b2-4134-a135-bb1a89c07efc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:22.885859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fba51-90b2-4134-a135-bb1a89c07efc None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:22.886710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fba51-90b2-4134-a135-bb1a89c07efc None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:22.887208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fba51-90b2-4134-a135-bb1a89c07efc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:22.888861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fba51-90b2-4134-a135-bb1a89c07efc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:22.898244 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 244/493] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:22.899417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fba51-90b2-4134-a135-bb1a89c07efc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:22.920113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51671eb5-1e66-4235-a6ec-9866e8e16a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:22.920662 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51671eb5-1e66-4235-a6ec-9866e8e16a9e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:22.920941 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51671eb5-1e66-4235-a6ec-9866e8e16a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:22.921213 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51671eb5-1e66-4235-a6ec-9866e8e16a9e None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:22.921913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51671eb5-1e66-4235-a6ec-9866e8e16a9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:22.922214 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51671eb5-1e66-4235-a6ec-9866e8e16a9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:22.924615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51671eb5-1e66-4235-a6ec-9866e8e16a9e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:22.940532 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 250/494] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:22.944156 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51671eb5-1e66-4235-a6ec-9866e8e16a9e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:23.002708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:23.015463 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fba51-90b2-4134-a135-bb1a89c07efc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:23.052519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4994e34-6831-4840-8a5d-7a9781d4ec2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:23.052737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51671eb5-1e66-4235-a6ec-9866e8e16a9e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:23.053146 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4994e34-6831-4840-8a5d-7a9781d4ec2f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:23.053424 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4994e34-6831-4840-8a5d-7a9781d4ec2f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:23.053696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4994e34-6831-4840-8a5d-7a9781d4ec2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:23.061946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:23.066394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4994e34-6831-4840-8a5d-7a9781d4ec2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:23.071717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-e4994e34-6831-4840-8a5d-7a9781d4ec2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:23.071717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:23.071717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:23.071717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:23.071717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:23.071717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:23.071717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:23.071717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:23.071717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:23.071717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:23.071717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:23.071717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:23.071717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:23.071717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:23.071717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:23.071717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:23.072841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:23.072841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:23.072841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:23.072841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:23.072841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:23.072841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:23.072841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:23.072841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:23.072841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:23.072841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:23.072841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:23.072841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:23.072841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:23.072841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:23.072841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:23.072841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:23.073650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:23.073650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:23.073650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:23.073650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:23.073650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:23.073650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:23.073650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:23.073650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:23.073650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:23.073650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:23.073650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:23.073650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:23.073650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 245/495] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:23.074287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4994e34-6831-4840-8a5d-7a9781d4ec2f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:23.095117 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762ec93-038b-4d21-aace-5c2ee797f318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:23.096013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762ec93-038b-4d21-aace-5c2ee797f318 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:23.096472 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762ec93-038b-4d21-aace-5c2ee797f318 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:23.096697 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762ec93-038b-4d21-aace-5c2ee797f318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:23.107402 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762ec93-038b-4d21-aace-5c2ee797f318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:23.108890 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762ec93-038b-4d21-aace-5c2ee797f318 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:23.126745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-0762ec93-038b-4d21-aace-5c2ee797f318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:23.126745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:23.126745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:23.126745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:23.126745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:23.126745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:23.126745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:23.126745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:23.126745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:23.126745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:23.126745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:23.126745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:23.126745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:23.126745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:23.126745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:23.126745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:23.127680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:23.128531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:23.128531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:23.128531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:23.128531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 251/496] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:23.129194 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762ec93-038b-4d21-aace-5c2ee797f318 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:23.186129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4994e34-6831-4840-8a5d-7a9781d4ec2f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:23.221309 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154b84e-cf9a-4e56-aadc-e37ff003e62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:23.221931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154b84e-cf9a-4e56-aadc-e37ff003e62f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:23.222209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154b84e-cf9a-4e56-aadc-e37ff003e62f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:23.222478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154b84e-cf9a-4e56-aadc-e37ff003e62f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:23.229930 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154b84e-cf9a-4e56-aadc-e37ff003e62f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:23.231358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154b84e-cf9a-4e56-aadc-e37ff003e62f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:23.239416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762ec93-038b-4d21-aace-5c2ee797f318 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:23.244127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-5154b84e-cf9a-4e56-aadc-e37ff003e62f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:42:23.244127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:23.244127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:23.244127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:23.244127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:23.244127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:23.244127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:23.244127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:23.244127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:23.244127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:23.244127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:23.244127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:23.244127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:23.244127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:23.244127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:23.244127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:23.245012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:23.245890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:23.245890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:23.245890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:23.245890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:23.245890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:23.245890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:23.245890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:23.245890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:23.245890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:42:23.245890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:23.245890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 246/497] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 7 07:42:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:23.246449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154b84e-cf9a-4e56-aadc-e37ff003e62f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:23.248545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:23.288560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107e95c-6f42-4cb9-a5d2-6f34c3af0fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:23.289109 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107e95c-6f42-4cb9-a5d2-6f34c3af0fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:23.289384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107e95c-6f42-4cb9-a5d2-6f34c3af0fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:23.289651 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107e95c-6f42-4cb9-a5d2-6f34c3af0fd2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:23.301468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107e95c-6f42-4cb9-a5d2-6f34c3af0fd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:23.307520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-7107e95c-6f42-4cb9-a5d2-6f34c3af0fd2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:23.307520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:23.307520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:23.307520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:23.307520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:23.307520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:23.307520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:23.307520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:23.307520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:23.307520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:23.307520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:23.307520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:23.307520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:23.307520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:23.307520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:23.307520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:23.308542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:23.308542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:23.308542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:23.308542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:23.308542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:23.308542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:23.308542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:23.308542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:23.308542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:23.308542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:23.308542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:23.308542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:23.308542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:23.308542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:23.308542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:23.308542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:23.309497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:23.309497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:23.309497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:23.309497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:23.309497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:23.309497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:23.309497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:23.309497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:23.309497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:23.309497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:23.309497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:23.309497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:23.309497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 252/498] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:23.312876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107e95c-6f42-4cb9-a5d2-6f34c3af0fd2 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:23.320990 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:23.365544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154b84e-cf9a-4e56-aadc-e37ff003e62f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:23.402252 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189624f5-cf07-4c7a-910a-73f85f947a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:23.403090 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189624f5-cf07-4c7a-910a-73f85f947a5d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:23.403365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189624f5-cf07-4c7a-910a-73f85f947a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:23.403631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189624f5-cf07-4c7a-910a-73f85f947a5d None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:23.404326 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189624f5-cf07-4c7a-910a-73f85f947a5d None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:23.404626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189624f5-cf07-4c7a-910a-73f85f947a5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:23.406012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189624f5-cf07-4c7a-910a-73f85f947a5d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:23.417888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 247/499] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:23.418692 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189624f5-cf07-4c7a-910a-73f85f947a5d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:23.426850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107e95c-6f42-4cb9-a5d2-6f34c3af0fd2 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:23.460441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d915af-9366-406d-b907-9c57c1464e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:23.461038 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d915af-9366-406d-b907-9c57c1464e09 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:23.461317 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d915af-9366-406d-b907-9c57c1464e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:23.461588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d915af-9366-406d-b907-9c57c1464e09 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:23.462214 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d915af-9366-406d-b907-9c57c1464e09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:23.462514 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d915af-9366-406d-b907-9c57c1464e09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:23.463896 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d915af-9366-406d-b907-9c57c1464e09 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:23.476168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 253/500] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:23.480637 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d915af-9366-406d-b907-9c57c1464e09 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:23.505124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:23.521560 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189624f5-cf07-4c7a-910a-73f85f947a5d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:23.543829 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:23.556552 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2824b21-e71a-4616-b433-e67d34abeca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:23.557145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2824b21-e71a-4616-b433-e67d34abeca7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:23.557421 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2824b21-e71a-4616-b433-e67d34abeca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:23.557687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2824b21-e71a-4616-b433-e67d34abeca7 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:23.558400 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2824b21-e71a-4616-b433-e67d34abeca7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:23.558702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2824b21-e71a-4616-b433-e67d34abeca7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:23.560163 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2824b21-e71a-4616-b433-e67d34abeca7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:23.577094 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d915af-9366-406d-b907-9c57c1464e09 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:23.596330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 248/501] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 7 07:42:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:23.597611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149cc23-abe5-4d47-b39b-c4035b411692 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:23.597898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149cc23-abe5-4d47-b39b-c4035b411692 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:23.598173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149cc23-abe5-4d47-b39b-c4035b411692 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:23.599380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 249/502] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:23.600060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149cc23-abe5-4d47-b39b-c4035b411692 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:23.616010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec046900-f3d3-4b94-a8b3-0963d4112f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:23.616633 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec046900-f3d3-4b94-a8b3-0963d4112f72 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:23.616911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec046900-f3d3-4b94-a8b3-0963d4112f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:23.617180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec046900-f3d3-4b94-a8b3-0963d4112f72 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:23.617835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec046900-f3d3-4b94-a8b3-0963d4112f72 None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:23.618134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec046900-f3d3-4b94-a8b3-0963d4112f72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:23.619498 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec046900-f3d3-4b94-a8b3-0963d4112f72 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:23.628222 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 254/503] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:23.628222 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec046900-f3d3-4b94-a8b3-0963d4112f72 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:23.705266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149cc23-abe5-4d47-b39b-c4035b411692 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:23.716006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:23.728817 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec046900-f3d3-4b94-a8b3-0963d4112f72 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:23.739394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d03679-634f-4225-8d55-67c2caf4f7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:23.740017 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d03679-634f-4225-8d55-67c2caf4f7cf None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:23.740327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d03679-634f-4225-8d55-67c2caf4f7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:23.740604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d03679-634f-4225-8d55-67c2caf4f7cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:23.750347 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d03679-634f-4225-8d55-67c2caf4f7cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:23.752340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d03679-634f-4225-8d55-67c2caf4f7cf None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:23.761769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceffc5f-e56a-4bb7-ba62-ca09eb1e9857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:23.762378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceffc5f-e56a-4bb7-ba62-ca09eb1e9857 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:23.762653 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceffc5f-e56a-4bb7-ba62-ca09eb1e9857 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:23.762950 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceffc5f-e56a-4bb7-ba62-ca09eb1e9857 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:23.766524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-74d03679-634f-4225-8d55-67c2caf4f7cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:23.766524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:23.766524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:23.766524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:23.766524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:23.766524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:23.766524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:23.766524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:23.766524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:23.766524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:23.766524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:23.766524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:23.766524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:23.766524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:23.766524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:23.766524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:23.768202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:23.769023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:23.769023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:23.769023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:23.769023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 250/504] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:23.769630 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d03679-634f-4225-8d55-67c2caf4f7cf None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:23.770975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceffc5f-e56a-4bb7-ba62-ca09eb1e9857 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:23.772451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceffc5f-e56a-4bb7-ba62-ca09eb1e9857 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:23.785605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-3ceffc5f-e56a-4bb7-ba62-ca09eb1e9857 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 07:42:23.785605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:23.785605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:23.785605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:23.785605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:23.785605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:23.785605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:23.785605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:23.785605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:23.785605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:23.785605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:23.785605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:23.785605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:23.785605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:23.785605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:23.785605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:23.786676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:23.787562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:23.787562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:23.787562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:23.787562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:23.787562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:23.787562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:23.787562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:23.787562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:23.787562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 07:42:23.787562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:23.787562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 255/505] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:23.788542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb6f84-f245-40e2-9240-2a2f3e8cf9ec None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:23.788542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb6f84-f245-40e2-9240-2a2f3e8cf9ec None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:23.788542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb6f84-f245-40e2-9240-2a2f3e8cf9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:23.806324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7ccb6f84-f245-40e2-9240-2a2f3e8cf9ec None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:23.807430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7ccb6f84-f245-40e2-9240-2a2f3e8cf9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:23.808014 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:23.872986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 256/506] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:23.873631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb6f84-f245-40e2-9240-2a2f3e8cf9ec None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:23.888985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d03679-634f-4225-8d55-67c2caf4f7cf None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:23.928142 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a2e7d-44b4-464c-a4a2-12ea8686eb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:23.928988 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a2e7d-44b4-464c-a4a2-12ea8686eb90 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:23.929407 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a2e7d-44b4-464c-a4a2-12ea8686eb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:23.929717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a2e7d-44b4-464c-a4a2-12ea8686eb90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:23.958561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:23.987275 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb6f84-f245-40e2-9240-2a2f3e8cf9ec None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:24.005432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a2e7d-44b4-464c-a4a2-12ea8686eb90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:24.007069 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a2e7d-44b4-464c-a4a2-12ea8686eb90 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:24.024904 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd48fe-4a33-4dba-9771-c5dfe3c82dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:24.025558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd48fe-4a33-4dba-9771-c5dfe3c82dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:24.025839 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd48fe-4a33-4dba-9771-c5dfe3c82dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:24.026111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd48fe-4a33-4dba-9771-c5dfe3c82dc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:24.037614 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd48fe-4a33-4dba-9771-c5dfe3c82dc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:24.039094 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd48fe-4a33-4dba-9771-c5dfe3c82dc0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:24.039301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:24.055995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-83cd48fe-4a33-4dba-9771-c5dfe3c82dc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:24.055995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:24.055995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:24.055995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:24.055995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:24.055995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:24.055995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:24.055995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:24.055995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:24.055995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:24.055995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:24.055995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:24.055995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:24.055995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:24.055995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:24.055995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:24.057089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:24.057921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:24.057921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:24.057921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:24.057921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 257/507] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:24.057921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd48fe-4a33-4dba-9771-c5dfe3c82dc0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:24.124172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 251/508] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:24.124823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a2e7d-44b4-464c-a4a2-12ea8686eb90 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:24.160003 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd48fe-4a33-4dba-9771-c5dfe3c82dc0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:24.187697 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:24.193698 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666df9a-b788-467e-ac56-9a8a86f04729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:24.194540 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666df9a-b788-467e-ac56-9a8a86f04729 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:24.194822 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666df9a-b788-467e-ac56-9a8a86f04729 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:24.195082 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666df9a-b788-467e-ac56-9a8a86f04729 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:24.195709 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666df9a-b788-467e-ac56-9a8a86f04729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:24.196002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666df9a-b788-467e-ac56-9a8a86f04729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:24.197378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666df9a-b788-467e-ac56-9a8a86f04729 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:24.209169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 258/509] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:24.209880 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666df9a-b788-467e-ac56-9a8a86f04729 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:24.225003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a2e7d-44b4-464c-a4a2-12ea8686eb90 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:24.259687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:24.265120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064280d-0197-4772-8d31-8c4ac15765d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:24.265719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064280d-0197-4772-8d31-8c4ac15765d5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:24.265994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064280d-0197-4772-8d31-8c4ac15765d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:24.266272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064280d-0197-4772-8d31-8c4ac15765d5 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:24.267027 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064280d-0197-4772-8d31-8c4ac15765d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:24.267328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064280d-0197-4772-8d31-8c4ac15765d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:24.268786 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064280d-0197-4772-8d31-8c4ac15765d5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:24.285825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 252/510] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Sep 7 07:42:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:24.286627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064280d-0197-4772-8d31-8c4ac15765d5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:24.309162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666df9a-b788-467e-ac56-9a8a86f04729 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:24.343510 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e26a55-84cd-4d94-9427-eb69702653d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:24.344234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e26a55-84cd-4d94-9427-eb69702653d3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:24.344510 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e26a55-84cd-4d94-9427-eb69702653d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:24.344812 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e26a55-84cd-4d94-9427-eb69702653d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:24.345703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e26a55-84cd-4d94-9427-eb69702653d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:24.346002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e26a55-84cd-4d94-9427-eb69702653d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:24.347356 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e26a55-84cd-4d94-9427-eb69702653d3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:24.358471 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 259/511] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:24.359173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e26a55-84cd-4d94-9427-eb69702653d3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:24.397433 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064280d-0197-4772-8d31-8c4ac15765d5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:24.408782 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:24.434364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ec653-f780-4aad-b1fa-174d6215a977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:24.434992 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ec653-f780-4aad-b1fa-174d6215a977 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:24.435272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ec653-f780-4aad-b1fa-174d6215a977 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:24.435548 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ec653-f780-4aad-b1fa-174d6215a977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:24.446869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ec653-f780-4aad-b1fa-174d6215a977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:24.448355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ec653-f780-4aad-b1fa-174d6215a977 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:24.462836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-cc7ec653-f780-4aad-b1fa-174d6215a977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:24.462836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:24.462836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:24.462836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:24.462836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:24.462836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:24.462836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:24.462836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:24.462836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:24.462836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:24.462836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:24.462836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:24.462836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:24.462836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:24.462836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:24.462836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:24.463904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:24.464768 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:24.464768 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:24.464768 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:24.464924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e26a55-84cd-4d94-9427-eb69702653d3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:24.465009 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 253/512] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:24.465009 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ec653-f780-4aad-b1fa-174d6215a977 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:24.498778 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5ac1d-e1e0-412c-8e9e-bb43724d5798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:24.499396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5ac1d-e1e0-412c-8e9e-bb43724d5798 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:24.499674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5ac1d-e1e0-412c-8e9e-bb43724d5798 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:24.499948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5ac1d-e1e0-412c-8e9e-bb43724d5798 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:24.509595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:24.511706 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea5ac1d-e1e0-412c-8e9e-bb43724d5798 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:24.513260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea5ac1d-e1e0-412c-8e9e-bb43724d5798 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:24.527359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-bea5ac1d-e1e0-412c-8e9e-bb43724d5798 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:24.527359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:24.527359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:24.527359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:24.527359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:24.527359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:24.527359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:24.527359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:24.527359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:24.527359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:24.527359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:24.527359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:24.527359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:24.527359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:24.527359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:24.527359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:24.528369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:24.529199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:24.529199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:24.529199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:24.529199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 260/513] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:24.533389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5ac1d-e1e0-412c-8e9e-bb43724d5798 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:24.567284 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ec653-f780-4aad-b1fa-174d6215a977 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:24.600257 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4690d8-909e-446e-862d-d1ff153aa08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:24.600928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4690d8-909e-446e-862d-d1ff153aa08f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:24.601207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4690d8-909e-446e-862d-d1ff153aa08f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:24.601502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4690d8-909e-446e-862d-d1ff153aa08f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:24.608695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4690d8-909e-446e-862d-d1ff153aa08f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:24.610127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4690d8-909e-446e-862d-d1ff153aa08f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:24.621803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-4d4690d8-909e-446e-862d-d1ff153aa08f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 07:42:24.621803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:24.621803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:24.621803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:24.621803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:24.621803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:24.621803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:24.621803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:24.621803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:24.621803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:24.621803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:24.621803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:24.621803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:24.621803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:24.621803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:24.621803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:24.622656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:24.623535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:24.623535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:24.623535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:24.623535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:24.623535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:24.623535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:24.623535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:24.623535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:24.623535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 07:42:24.623535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:24.623535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 254/514] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:24.624105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4690d8-909e-446e-862d-d1ff153aa08f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:24.636890 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5ac1d-e1e0-412c-8e9e-bb43724d5798 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:24.638410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:24.670373 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd80c7-9911-40b7-81d5-97dfc4897619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:24.670964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd80c7-9911-40b7-81d5-97dfc4897619 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:24.671238 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd80c7-9911-40b7-81d5-97dfc4897619 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:24.671503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd80c7-9911-40b7-81d5-97dfc4897619 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:24.722945 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4690d8-909e-446e-862d-d1ff153aa08f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:24.724481 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:24.740576 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd80c7-9911-40b7-81d5-97dfc4897619 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:24.742049 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd80c7-9911-40b7-81d5-97dfc4897619 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:24.757819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a90d6f-312f-4654-ab10-29bac2fbd3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:24.758437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a90d6f-312f-4654-ab10-29bac2fbd3cc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:24.758717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a90d6f-312f-4654-ab10-29bac2fbd3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:24.758995 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a90d6f-312f-4654-ab10-29bac2fbd3cc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:24.766227 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a90d6f-312f-4654-ab10-29bac2fbd3cc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:24.768005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a90d6f-312f-4654-ab10-29bac2fbd3cc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:24.778978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f8a90d6f-312f-4654-ab10-29bac2fbd3cc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 07:42:24.778978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:24.778978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:24.778978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:24.778978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:24.778978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:24.778978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:24.778978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:24.778978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:24.778978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:24.778978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:24.778978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:24.778978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:24.778978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:24.778978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:24.778978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:24.779869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:24.781098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:24.781098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:24.781098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:24.781098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:24.781098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:24.781098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:24.781098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:24.781098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:24.781098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 07:42:24.781098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:24.781098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 255/515] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:24.783343 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a90d6f-312f-4654-ab10-29bac2fbd3cc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:24.839771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 261/516] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:24.840582 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd80c7-9911-40b7-81d5-97dfc4897619 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:24.842784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:24.889212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a90d6f-312f-4654-ab10-29bac2fbd3cc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:24.924157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159e2f1-8ae8-468d-959f-bbc53171a3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:24.924742 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159e2f1-8ae8-468d-959f-bbc53171a3dc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:24.925026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159e2f1-8ae8-468d-959f-bbc53171a3dc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:24.925288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159e2f1-8ae8-468d-959f-bbc53171a3dc None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:24.925908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159e2f1-8ae8-468d-959f-bbc53171a3dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:24.926202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159e2f1-8ae8-468d-959f-bbc53171a3dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:24.927545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159e2f1-8ae8-468d-959f-bbc53171a3dc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:24.938287 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd80c7-9911-40b7-81d5-97dfc4897619 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:24.939252 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 256/517] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:24.940040 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159e2f1-8ae8-468d-959f-bbc53171a3dc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:24.963346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:24.972484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe350c-f20c-43b1-849f-a09ed44a09b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:24.973083 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe350c-f20c-43b1-849f-a09ed44a09b8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:24.973357 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe350c-f20c-43b1-849f-a09ed44a09b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:24.973625 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe350c-f20c-43b1-849f-a09ed44a09b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:24.974252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe350c-f20c-43b1-849f-a09ed44a09b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:24.974556 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe350c-f20c-43b1-849f-a09ed44a09b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:24.975911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe350c-f20c-43b1-849f-a09ed44a09b8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:24.986998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 262/518] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:24.987774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe350c-f20c-43b1-849f-a09ed44a09b8 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:25.037298 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159e2f1-8ae8-468d-959f-bbc53171a3dc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:25.048979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:25.070560 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e805c-9d67-4331-b5bd-5681146517c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:25.071157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e805c-9d67-4331-b5bd-5681146517c4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:25.071434 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e805c-9d67-4331-b5bd-5681146517c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:25.071701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e805c-9d67-4331-b5bd-5681146517c4 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:25.072442 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e805c-9d67-4331-b5bd-5681146517c4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:25.072742 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e805c-9d67-4331-b5bd-5681146517c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:25.074138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e805c-9d67-4331-b5bd-5681146517c4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:25.088700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe350c-f20c-43b1-849f-a09ed44a09b8 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:25.090662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 257/519] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Sep 7 07:42:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:25.091239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e805c-9d67-4331-b5bd-5681146517c4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:25.122190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30445201-6751-4de2-9a91-8e91935c42be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:25.122783 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30445201-6751-4de2-9a91-8e91935c42be None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:25.123055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30445201-6751-4de2-9a91-8e91935c42be None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:25.123322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30445201-6751-4de2-9a91-8e91935c42be None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:25.124044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30445201-6751-4de2-9a91-8e91935c42be None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:25.124374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30445201-6751-4de2-9a91-8e91935c42be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:25.125787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30445201-6751-4de2-9a91-8e91935c42be None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:25.161736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 263/520] 10.222.0.26 () {56 vars in 1139 bytes} [Wed Sep 7 07:42:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:25.162548 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30445201-6751-4de2-9a91-8e91935c42be None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:25.173612 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:25.193063 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e805c-9d67-4331-b5bd-5681146517c4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:25.225927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e9513-3bcb-49dc-8c7b-fdcac86b05e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:25.226535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e9513-3bcb-49dc-8c7b-fdcac86b05e0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:25.226807 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e9513-3bcb-49dc-8c7b-fdcac86b05e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:25.227076 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e9513-3bcb-49dc-8c7b-fdcac86b05e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:25.237738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e9513-3bcb-49dc-8c7b-fdcac86b05e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:25.239171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e9513-3bcb-49dc-8c7b-fdcac86b05e0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:25.253798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-b30e9513-3bcb-49dc-8c7b-fdcac86b05e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:25.253798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:25.253798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:25.253798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:25.253798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:25.253798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:25.253798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:25.253798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:25.253798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:25.253798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:25.253798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:25.253798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:25.253798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:25.253798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:25.253798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:25.253798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:25.254860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:25.255693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:25.255693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:25.255693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:25.255693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 258/521] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:25.255909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e9513-3bcb-49dc-8c7b-fdcac86b05e0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:25.266460 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30445201-6751-4de2-9a91-8e91935c42be None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:25.290835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:25.305689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f10722-a2db-4469-94c4-e762628ecb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:25.306306 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f10722-a2db-4469-94c4-e762628ecb8f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:25.306578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f10722-a2db-4469-94c4-e762628ecb8f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:25.306843 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f10722-a2db-4469-94c4-e762628ecb8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:25.313502 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f10722-a2db-4469-94c4-e762628ecb8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:25.315549 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f10722-a2db-4469-94c4-e762628ecb8f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:25.327391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-f6f10722-a2db-4469-94c4-e762628ecb8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:25.327391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:25.327391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:25.327391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:25.327391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:25.327391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:25.327391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:25.327391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:25.327391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:25.327391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:25.327391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:25.327391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:25.327391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:25.327391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:25.327391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:25.328536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:25.329710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:25.329710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:25.329710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:25.329710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:25.329710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:25.329710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:25.329710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:25.329710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:25.332415 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 264/522] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:25.333546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f10722-a2db-4469-94c4-e762628ecb8f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:25.354683 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e9513-3bcb-49dc-8c7b-fdcac86b05e0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:25.383613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:25.388716 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76783d1e-0449-4544-aa64-4274a69d255f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:25.389315 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76783d1e-0449-4544-aa64-4274a69d255f None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:25.389594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76783d1e-0449-4544-aa64-4274a69d255f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:25.389866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76783d1e-0449-4544-aa64-4274a69d255f None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:25.390506 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76783d1e-0449-4544-aa64-4274a69d255f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:25.392477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76783d1e-0449-4544-aa64-4274a69d255f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:25.414161 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 259/523] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:42:25 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:42:25.414161 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76783d1e-0449-4544-aa64-4274a69d255f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:25.456840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f10722-a2db-4469-94c4-e762628ecb8f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:25.497590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85010c-43b6-4bfa-b131-fbb4cb416876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:25.498205 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85010c-43b6-4bfa-b131-fbb4cb416876 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:25.498480 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85010c-43b6-4bfa-b131-fbb4cb416876 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:25.498758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85010c-43b6-4bfa-b131-fbb4cb416876 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:25.527936 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76783d1e-0449-4544-aa64-4274a69d255f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:25.556580 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:25.571646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395c90a-6a92-472c-b0cc-87a95cdddef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:25.572302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395c90a-6a92-472c-b0cc-87a95cdddef5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:25.572585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395c90a-6a92-472c-b0cc-87a95cdddef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:25.572864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395c90a-6a92-472c-b0cc-87a95cdddef5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:25.582187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85010c-43b6-4bfa-b131-fbb4cb416876 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:25.583671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85010c-43b6-4bfa-b131-fbb4cb416876 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:25.636724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:25.650532 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395c90a-6a92-472c-b0cc-87a95cdddef5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:25.651991 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395c90a-6a92-472c-b0cc-87a95cdddef5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:25.704268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 265/524] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:25.704268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85010c-43b6-4bfa-b131-fbb4cb416876 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:25.759581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 260/525] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:25.760454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395c90a-6a92-472c-b0cc-87a95cdddef5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:25.806348 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:25.827299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85010c-43b6-4bfa-b131-fbb4cb416876 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:25.850811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:25.865361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395c90a-6a92-472c-b0cc-87a95cdddef5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:25.865739 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc8849-1953-47c0-a513-bd1b90fc110b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:25.866346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc8849-1953-47c0-a513-bd1b90fc110b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:25.866624 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc8849-1953-47c0-a513-bd1b90fc110b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:25.866896 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc8849-1953-47c0-a513-bd1b90fc110b None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:25.867690 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc8849-1953-47c0-a513-bd1b90fc110b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:25.868021 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc8849-1953-47c0-a513-bd1b90fc110b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:25.870230 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc8849-1953-47c0-a513-bd1b90fc110b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:25.884403 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 266/526] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:25.884403 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc8849-1953-47c0-a513-bd1b90fc110b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:25.919825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80322705-c177-45ad-9005-82920db81fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:25.920386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80322705-c177-45ad-9005-82920db81fcf None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:25.920663 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80322705-c177-45ad-9005-82920db81fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:25.920929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80322705-c177-45ad-9005-82920db81fcf None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:25.921624 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80322705-c177-45ad-9005-82920db81fcf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:25.921927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80322705-c177-45ad-9005-82920db81fcf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:25.924230 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80322705-c177-45ad-9005-82920db81fcf None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:25.954493 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 261/527] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:25.954493 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80322705-c177-45ad-9005-82920db81fcf None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:26.008300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc8849-1953-47c0-a513-bd1b90fc110b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:26.042727 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d062a3-842a-483d-b223-a9c256f9a949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:26.043356 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d062a3-842a-483d-b223-a9c256f9a949 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:26.043630 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d062a3-842a-483d-b223-a9c256f9a949 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:26.043897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d062a3-842a-483d-b223-a9c256f9a949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:26.053343 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:26.056699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d062a3-842a-483d-b223-a9c256f9a949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:26.058892 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d062a3-842a-483d-b223-a9c256f9a949 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:26.059130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80322705-c177-45ad-9005-82920db81fcf None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:26.076601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-b1d062a3-842a-483d-b223-a9c256f9a949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:26.076601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:26.076601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:26.076601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:26.076601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:26.076601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:26.076601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:26.076601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:26.076601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:26.076601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:26.076601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:26.076601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:26.076601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:26.076601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:26.076601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:26.076601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:26.077632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:26.078454 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:26.078454 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:26.078454 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:26.078454 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 267/528] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:26.079208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d062a3-842a-483d-b223-a9c256f9a949 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:26.103935 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958fc42-3803-487c-9caf-87c099c31308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:26.104798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958fc42-3803-487c-9caf-87c099c31308 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:26.105082 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958fc42-3803-487c-9caf-87c099c31308 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:26.105719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958fc42-3803-487c-9caf-87c099c31308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:26.118792 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958fc42-3803-487c-9caf-87c099c31308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:26.120668 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958fc42-3803-487c-9caf-87c099c31308 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:26.132260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:26.140789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-9958fc42-3803-487c-9caf-87c099c31308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:26.140789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:26.140789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:26.140789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:26.140789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:26.140789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:26.140789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:26.140789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:26.140789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:26.140789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:26.140789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:26.140789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:26.140789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:26.140789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:26.140789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:26.140789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:26.142401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:26.143397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:26.143397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:26.143397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:26.150660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 262/529] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:26.151243 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958fc42-3803-487c-9caf-87c099c31308 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:26.199457 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d062a3-842a-483d-b223-a9c256f9a949 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:26.237183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c551f7-be33-4cc0-b7de-33829aacec55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:26.237927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c551f7-be33-4cc0-b7de-33829aacec55 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:26.238254 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c551f7-be33-4cc0-b7de-33829aacec55 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:26.238626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c551f7-be33-4cc0-b7de-33829aacec55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:26.246139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c551f7-be33-4cc0-b7de-33829aacec55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:26.247620 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c551f7-be33-4cc0-b7de-33829aacec55 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:26.259269 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958fc42-3803-487c-9caf-87c099c31308 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:26.262315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-b2c551f7-be33-4cc0-b7de-33829aacec55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:26.262315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:26.262315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:26.262315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:26.262315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:26.262315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:26.262315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:26.262315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:26.262315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:26.262315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:26.262315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:26.262315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:26.262315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:26.262315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:26.262315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:26.263148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:26.263980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:26.263980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:26.263980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:26.263980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:26.263980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:26.263980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:26.263980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:26.263980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:26.263980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 268/530] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:26.264901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c551f7-be33-4cc0-b7de-33829aacec55 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:26.299514 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3403fb1-1081-4f12-8072-84bef03980c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:26.300128 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3403fb1-1081-4f12-8072-84bef03980c2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:26.300409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3403fb1-1081-4f12-8072-84bef03980c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:26.300678 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3403fb1-1081-4f12-8072-84bef03980c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:26.301404 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3403fb1-1081-4f12-8072-84bef03980c2 None admin] RBAC: Adding query filter params (role.id=e7960e6d957f4a1fa6547565db026f24, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=81d5689e9a344bafb74ca423ffd2b42a) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:26.301709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3403fb1-1081-4f12-8072-84bef03980c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:26.303129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3403fb1-1081-4f12-8072-84bef03980c2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:26.316124 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 263/531] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:42:26 2022] GET /identity/v3/role_assignments?user.id=81d5689e9a344bafb74ca423ffd2b42a&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=e7960e6d957f4a1fa6547565db026f24 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:26.316124 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3403fb1-1081-4f12-8072-84bef03980c2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:26.335803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:26.390202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:26.411078 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c551f7-be33-4cc0-b7de-33829aacec55 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:26.446671 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3403fb1-1081-4f12-8072-84bef03980c2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:26.453897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e41e64-ce52-4152-9c1c-b93420d88767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:26.454474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e41e64-ce52-4152-9c1c-b93420d88767 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:26.454751 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e41e64-ce52-4152-9c1c-b93420d88767 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:26.455009 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e41e64-ce52-4152-9c1c-b93420d88767 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:26.455639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e41e64-ce52-4152-9c1c-b93420d88767 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:26.455934 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e41e64-ce52-4152-9c1c-b93420d88767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:26.457470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e41e64-ce52-4152-9c1c-b93420d88767 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:26.471693 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 269/532] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:26.472683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e41e64-ce52-4152-9c1c-b93420d88767 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:26.497612 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187598c-fd2d-4414-baea-adc36b854ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:26.498924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187598c-fd2d-4414-baea-adc36b854ddc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:26.499407 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187598c-fd2d-4414-baea-adc36b854ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:26.500037 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187598c-fd2d-4414-baea-adc36b854ddc None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:26.501319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187598c-fd2d-4414-baea-adc36b854ddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:26.502428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187598c-fd2d-4414-baea-adc36b854ddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:26.504539 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187598c-fd2d-4414-baea-adc36b854ddc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:26.520180 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 264/533] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:26.521801 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187598c-fd2d-4414-baea-adc36b854ddc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:26.599995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e41e64-ce52-4152-9c1c-b93420d88767 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:26.618417 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:26.642651 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187598c-fd2d-4414-baea-adc36b854ddc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:26.649653 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f775d6e-3769-4134-9f53-bbf09772dca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:26.650520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f775d6e-3769-4134-9f53-bbf09772dca7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:26.651535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f775d6e-3769-4134-9f53-bbf09772dca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:26.652039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f775d6e-3769-4134-9f53-bbf09772dca7 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:26.652995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f775d6e-3769-4134-9f53-bbf09772dca7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:26.653982 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f775d6e-3769-4134-9f53-bbf09772dca7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:26.656354 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f775d6e-3769-4134-9f53-bbf09772dca7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:26.669203 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:26.682769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ea545-a342-40a8-8516-678f89837114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:26.683398 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ea545-a342-40a8-8516-678f89837114 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:26.683672 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ea545-a342-40a8-8516-678f89837114 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:26.683947 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ea545-a342-40a8-8516-678f89837114 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:26.684048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 270/534] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:26.684876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f775d6e-3769-4134-9f53-bbf09772dca7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:26.692737 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ea545-a342-40a8-8516-678f89837114 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:26.694214 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ea545-a342-40a8-8516-678f89837114 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:26.708974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d55ea545-a342-40a8-8516-678f89837114 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:26.708974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:26.708974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:26.708974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:26.708974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:26.708974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:26.708974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:26.708974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:26.708974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:26.708974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:26.708974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:26.708974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:26.708974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:26.708974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:26.708974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:26.709820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:26.710628 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:26.710628 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:26.710628 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:26.710628 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:26.710628 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:26.710628 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:26.710628 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:26.710628 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:26.714955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 265/535] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:26.715871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ea545-a342-40a8-8516-678f89837114 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:26.807754 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f775d6e-3769-4134-9f53-bbf09772dca7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:26.833458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ea545-a342-40a8-8516-678f89837114 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:26.842714 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325f181-a495-43d2-bf78-71c09647db6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:26.843341 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325f181-a495-43d2-bf78-71c09647db6d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:26.843620 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325f181-a495-43d2-bf78-71c09647db6d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:26.843890 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325f181-a495-43d2-bf78-71c09647db6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:26.850992 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325f181-a495-43d2-bf78-71c09647db6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:26.852453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325f181-a495-43d2-bf78-71c09647db6d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:26.864323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-d325f181-a495-43d2-bf78-71c09647db6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:26.864323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:26.864323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:26.864323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:26.864323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:26.864323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:26.864323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:26.864323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:26.864323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:26.864323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:26.864323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:26.864323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:26.864323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:26.864323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:26.864323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:26.865245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:26.866079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:26.866079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:26.866079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:26.866079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:26.866079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:26.866079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:26.866079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:26.866079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:26.870131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 271/536] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:26.870964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325f181-a495-43d2-bf78-71c09647db6d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:26.877524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ceb9f-017c-4349-a57e-73362e75631a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:26.878439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ceb9f-017c-4349-a57e-73362e75631a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:26.878923 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ceb9f-017c-4349-a57e-73362e75631a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:26.879399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ceb9f-017c-4349-a57e-73362e75631a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:26.880530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ceb9f-017c-4349-a57e-73362e75631a None admin] RBAC: Adding query filter params (role.id=e7960e6d957f4a1fa6547565db026f24, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=9b0652c89c9945e2866512c9da4793ba) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:26.881091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ceb9f-017c-4349-a57e-73362e75631a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:26.882866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ceb9f-017c-4349-a57e-73362e75631a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:26.889629 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 266/537] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:42:26 2022] GET /identity/v3/role_assignments?user.id=9b0652c89c9945e2866512c9da4793ba&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=e7960e6d957f4a1fa6547565db026f24 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:26.890931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ceb9f-017c-4349-a57e-73362e75631a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:26.904809 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:26.958217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:27.022915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ceb9f-017c-4349-a57e-73362e75631a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:27.068139 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b4bb7-dba0-4322-bf13-c3d0d8ed582f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:27.069536 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b4bb7-dba0-4322-bf13-c3d0d8ed582f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:27.071370 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b4bb7-dba0-4322-bf13-c3d0d8ed582f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:27.072504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b4bb7-dba0-4322-bf13-c3d0d8ed582f None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:27.073477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b4bb7-dba0-4322-bf13-c3d0d8ed582f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:27.074315 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b4bb7-dba0-4322-bf13-c3d0d8ed582f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:27.076724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b4bb7-dba0-4322-bf13-c3d0d8ed582f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:27.085808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325f181-a495-43d2-bf78-71c09647db6d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:27.091684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 267/538] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:27.100029 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b4bb7-dba0-4322-bf13-c3d0d8ed582f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:27.134269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25996a9b-f096-49bf-bf46-8e34012e754c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:27.134862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996a9b-f096-49bf-bf46-8e34012e754c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:27.135182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996a9b-f096-49bf-bf46-8e34012e754c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:27.135454 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996a9b-f096-49bf-bf46-8e34012e754c None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:27.136155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25996a9b-f096-49bf-bf46-8e34012e754c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:27.136458 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25996a9b-f096-49bf-bf46-8e34012e754c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:27.138045 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25996a9b-f096-49bf-bf46-8e34012e754c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:27.153511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 272/539] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:27.165157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25996a9b-f096-49bf-bf46-8e34012e754c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:27.229557 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b4bb7-dba0-4322-bf13-c3d0d8ed582f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:27.248147 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:27.281216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:27.294095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25996a9b-f096-49bf-bf46-8e34012e754c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:27.312709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2881e-8e05-401b-a622-08f401331b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:27.313368 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2881e-8e05-401b-a622-08f401331b5e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:27.313835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2881e-8e05-401b-a622-08f401331b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:27.314191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2881e-8e05-401b-a622-08f401331b5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:27.314929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb2881e-8e05-401b-a622-08f401331b5e None admin] RBAC: Adding query filter params (role.id=e7960e6d957f4a1fa6547565db026f24, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=330a38822d33439191773e9a8b4cbe04) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:27.315245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb2881e-8e05-401b-a622-08f401331b5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:27.316708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb2881e-8e05-401b-a622-08f401331b5e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:27.340239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 268/540] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:42:27 2022] GET /identity/v3/role_assignments?user.id=330a38822d33439191773e9a8b4cbe04&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=e7960e6d957f4a1fa6547565db026f24 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:27.341405 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c6bf0-bc12-44b1-992d-2f347b9bc6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:27.342231 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c6bf0-bc12-44b1-992d-2f347b9bc6fa None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:27.342689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c6bf0-bc12-44b1-992d-2f347b9bc6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:27.343130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c6bf0-bc12-44b1-992d-2f347b9bc6fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:27.344051 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c6bf0-bc12-44b1-992d-2f347b9bc6fa None admin] RBAC: Adding query filter params (role.id=e7960e6d957f4a1fa6547565db026f24, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=92699b512c064b05948afe597650fdb6) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:27.344555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c6bf0-bc12-44b1-992d-2f347b9bc6fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:27.347181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c6bf0-bc12-44b1-992d-2f347b9bc6fa None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:27.366345 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 273/541] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:42:27 2022] GET /identity/v3/role_assignments?user.id=92699b512c064b05948afe597650fdb6&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=e7960e6d957f4a1fa6547565db026f24 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:27.869919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c47a0c-2583-4107-9049-4bef33289644 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:27.870937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c47a0c-2583-4107-9049-4bef33289644 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:27.871394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c47a0c-2583-4107-9049-4bef33289644 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:27.873159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 269/542] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:27.881868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119990d5-d29d-447d-8809-cafbf54d05fa None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:27.882675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119990d5-d29d-447d-8809-cafbf54d05fa None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:27.883152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119990d5-d29d-447d-8809-cafbf54d05fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:27.912003 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-119990d5-d29d-447d-8809-cafbf54d05fa None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:27.913536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-119990d5-d29d-447d-8809-cafbf54d05fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:28.001586 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 274/543] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:42:28.342282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c47a0c-2583-4107-9049-4bef33289644 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:28.490442 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c47a0c-2583-4107-9049-4bef33289644 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:28.540009 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0a067-6b0f-484b-afa2-4981aa22df04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:28.540659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0a067-6b0f-484b-afa2-4981aa22df04 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:28.540935 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0a067-6b0f-484b-afa2-4981aa22df04 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:28.541257 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0a067-6b0f-484b-afa2-4981aa22df04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:28.556927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa0a067-6b0f-484b-afa2-4981aa22df04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:28.562274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-1aa0a067-6b0f-484b-afa2-4981aa22df04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:28.562274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:28.562274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:28.562274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:28.562274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:28.562274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:28.562274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:28.562274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:28.562274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:28.562274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:28.562274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:28.562274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:28.562274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:28.562274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:28.562274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:28.562274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:28.563162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:28.563162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:28.563162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:28.563162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:28.563162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:28.563162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:28.563162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:28.563162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:28.563162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:28.563162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:28.563162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:28.563162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:28.563162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:28.563162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:28.563162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:28.563162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:28.564537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:28.564537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:28.564537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:28.564537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:28.564537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:28.564537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:28.564537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:28.564537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:28.564537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:28.564537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:28.564537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:28.564537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:28.564537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 270/544] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:28.571737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119990d5-d29d-447d-8809-cafbf54d05fa None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:28.724744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:28.745786 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119990d5-d29d-447d-8809-cafbf54d05fa None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:28.771978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4f6f0-9818-4fb4-8f06-6e9ce52e41c2 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:28.772290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4f6f0-9818-4fb4-8f06-6e9ce52e41c2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:28.772569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4f6f0-9818-4fb4-8f06-6e9ce52e41c2 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:28.774029 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 271/545] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:28.780802 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4b58b-2116-48e8-8b5b-ff77d9a82994 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:28.781082 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4b58b-2116-48e8-8b5b-ff77d9a82994 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:28.781358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4b58b-2116-48e8-8b5b-ff77d9a82994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:28.802835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2600d-278e-4741-9ddb-f3c35dd5cc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:28.803783 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2600d-278e-4741-9ddb-f3c35dd5cc56 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:28.804259 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2600d-278e-4741-9ddb-f3c35dd5cc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:28.804363 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4bc4b58b-2116-48e8-8b5b-ff77d9a82994 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:28.804793 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2600d-278e-4741-9ddb-f3c35dd5cc56 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:28.805493 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4bc4b58b-2116-48e8-8b5b-ff77d9a82994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:28.805770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2600d-278e-4741-9ddb-f3c35dd5cc56 None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:28.806204 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2600d-278e-4741-9ddb-f3c35dd5cc56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:28.807711 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2600d-278e-4741-9ddb-f3c35dd5cc56 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:28.826488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 275/546] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:28.831908 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2600d-278e-4741-9ddb-f3c35dd5cc56 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:28.842374 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:28.930236 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 272/547] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:42:29.022945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2600d-278e-4741-9ddb-f3c35dd5cc56 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:29.073898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434bc3f-ba2f-4f4d-9b13-93e6666df49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:29.074533 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434bc3f-ba2f-4f4d-9b13-93e6666df49b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:29.074813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434bc3f-ba2f-4f4d-9b13-93e6666df49b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:29.075085 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434bc3f-ba2f-4f4d-9b13-93e6666df49b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:29.086416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3434bc3f-ba2f-4f4d-9b13-93e6666df49b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:29.087882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3434bc3f-ba2f-4f4d-9b13-93e6666df49b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:29.096685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:29.103310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-3434bc3f-ba2f-4f4d-9b13-93e6666df49b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:29.103310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:29.103310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:29.103310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:29.103310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:29.103310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:29.103310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:29.103310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:29.103310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:29.103310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:29.103310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:29.103310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:29.103310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:29.103310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:29.103310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:29.103310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:29.104215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:29.105361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:29.105361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:29.105361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:29.105361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 276/548] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:42:29.110012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4b58b-2116-48e8-8b5b-ff77d9a82994 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:29.254639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434bc3f-ba2f-4f4d-9b13-93e6666df49b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:29.273612 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4b58b-2116-48e8-8b5b-ff77d9a82994 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:29.340097 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb17142-6ea2-4375-977a-d771b3975297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:29.340711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb17142-6ea2-4375-977a-d771b3975297 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:29.340979 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb17142-6ea2-4375-977a-d771b3975297 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:29.341390 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb17142-6ea2-4375-977a-d771b3975297 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:29.342031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb17142-6ea2-4375-977a-d771b3975297 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:29.342476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb17142-6ea2-4375-977a-d771b3975297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:29.343839 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb17142-6ea2-4375-977a-d771b3975297 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:29.366694 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 273/549] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:29.371959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb17142-6ea2-4375-977a-d771b3975297 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:29.381698 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434bc3f-ba2f-4f4d-9b13-93e6666df49b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:29.420006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6e51d-5f7c-45f7-8ab6-16bf8ad5ab15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:29.420666 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6e51d-5f7c-45f7-8ab6-16bf8ad5ab15 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:29.420944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6e51d-5f7c-45f7-8ab6-16bf8ad5ab15 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:29.421214 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6e51d-5f7c-45f7-8ab6-16bf8ad5ab15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:29.425207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:29.434881 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6e51d-5f7c-45f7-8ab6-16bf8ad5ab15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:29.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-ced6e51d-5f7c-45f7-8ab6-16bf8ad5ab15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:29.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:29.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:29.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:29.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:29.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:29.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:29.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:29.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:29.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:29.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:29.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:29.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:29.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:29.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:29.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:29.444845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:29.444845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:29.444845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:29.444845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:29.444845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:29.444845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:29.444845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:29.444845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:29.444845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:29.444845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:29.444845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:29.444845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:29.444845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:29.444845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:29.444845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:29.444845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:29.445986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:29.445986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:29.445986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:29.445986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:29.445986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:29.445986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:29.445986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:29.445986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:29.445986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:29.445986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:29.445986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:29.445986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:29.448797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 277/550] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:29.450638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6e51d-5f7c-45f7-8ab6-16bf8ad5ab15 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:29.536687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:29.538855 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb17142-6ea2-4375-977a-d771b3975297 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:29.569109 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6e51d-5f7c-45f7-8ab6-16bf8ad5ab15 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:29.610049 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b94a0-d325-4a6a-abc5-5faeaac9e836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:29.610804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b94a0-d325-4a6a-abc5-5faeaac9e836 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:29.611090 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b94a0-d325-4a6a-abc5-5faeaac9e836 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:29.611365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b94a0-d325-4a6a-abc5-5faeaac9e836 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:29.612031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b94a0-d325-4a6a-abc5-5faeaac9e836 None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:29.612368 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b94a0-d325-4a6a-abc5-5faeaac9e836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:29.613758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b94a0-d325-4a6a-abc5-5faeaac9e836 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:29.623282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 278/551] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:29.629350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b94a0-d325-4a6a-abc5-5faeaac9e836 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:29.706386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc692c-169f-4620-b5aa-cf29fdb5377d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:29.707199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc692c-169f-4620-b5aa-cf29fdb5377d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:29.707595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc692c-169f-4620-b5aa-cf29fdb5377d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:29.707965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc692c-169f-4620-b5aa-cf29fdb5377d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:29.718697 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc692c-169f-4620-b5aa-cf29fdb5377d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:29.720856 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc692c-169f-4620-b5aa-cf29fdb5377d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:29.737933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-fbbc692c-169f-4620-b5aa-cf29fdb5377d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 07:42:29.737933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:29.737933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:29.737933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:29.737933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:29.737933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:29.737933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:29.737933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:29.737933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:29.737933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:29.737933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:29.737933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:29.737933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:29.737933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:29.737933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:29.737933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:29.739081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:29.739956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:29.739956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:29.739956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:29.739956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:29.739956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:29.739956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:29.739956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:29.739956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:29.739956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 07:42:29.739956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:29.740493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b94a0-d325-4a6a-abc5-5faeaac9e836 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:29.743997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 274/552] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:42:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:42:29.746323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de84ef0-2560-45ee-8e23-b9eb4b3bf57f None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:29.746606 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de84ef0-2560-45ee-8e23-b9eb4b3bf57f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:29.746881 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de84ef0-2560-45ee-8e23-b9eb4b3bf57f None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:29.751272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 275/553] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:29.752354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de84ef0-2560-45ee-8e23-b9eb4b3bf57f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:29.772972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:29.788494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee78075-ed8d-468b-9466-deffb876ea70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:29.789325 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee78075-ed8d-468b-9466-deffb876ea70 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:29.790009 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee78075-ed8d-468b-9466-deffb876ea70 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:29.790523 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee78075-ed8d-468b-9466-deffb876ea70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:29.806708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee78075-ed8d-468b-9466-deffb876ea70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:29.808348 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee78075-ed8d-468b-9466-deffb876ea70 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:29.828820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-9ee78075-ed8d-468b-9466-deffb876ea70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:29.828820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:29.828820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:29.828820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:29.828820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:29.828820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:29.828820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:29.828820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:29.828820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:29.828820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:29.828820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:29.828820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:29.828820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:29.828820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:29.828820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:29.828820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:29.831475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:29.833262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:29.833262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:29.833262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:29.840112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 279/554] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:42:29.840112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29371304-e414-4ac2-9157-cc9e8c53e1f0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:29.840112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29371304-e414-4ac2-9157-cc9e8c53e1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:29.840112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29371304-e414-4ac2-9157-cc9e8c53e1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:29.866410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-29371304-e414-4ac2-9157-cc9e8c53e1f0 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:29.867838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-29371304-e414-4ac2-9157-cc9e8c53e1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:29.868275 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:29.888375 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de84ef0-2560-45ee-8e23-b9eb4b3bf57f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:29.924631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7678715-81b8-4eb4-9717-c41fe339ba9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:29.925241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7678715-81b8-4eb4-9717-c41fe339ba9a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:29.925602 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7678715-81b8-4eb4-9717-c41fe339ba9a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:29.925879 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7678715-81b8-4eb4-9717-c41fe339ba9a None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:29.926616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7678715-81b8-4eb4-9717-c41fe339ba9a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:29.926919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7678715-81b8-4eb4-9717-c41fe339ba9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:29.928379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7678715-81b8-4eb4-9717-c41fe339ba9a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:29.945538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 280/555] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:29.946455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29371304-e414-4ac2-9157-cc9e8c53e1f0 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:29.956112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 276/556] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Sep 7 07:42:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:29.957593 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b51bc-73f8-41cb-9470-15c7df2a49cc None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:29.957971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b51bc-73f8-41cb-9470-15c7df2a49cc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:29.958397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b51bc-73f8-41cb-9470-15c7df2a49cc None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:29.959996 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 277/557] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:29.965559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed684af4-3960-4cef-b0d6-9fc93b9898e9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:29.965843 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed684af4-3960-4cef-b0d6-9fc93b9898e9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:29.966464 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed684af4-3960-4cef-b0d6-9fc93b9898e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:29.990311 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ed684af4-3960-4cef-b0d6-9fc93b9898e9 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:29.991774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ed684af4-3960-4cef-b0d6-9fc93b9898e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:30.065493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29371304-e414-4ac2-9157-cc9e8c53e1f0 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:30.068697 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:30.085998 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 278/558] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:30.086922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed684af4-3960-4cef-b0d6-9fc93b9898e9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:30.131529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc10483-b425-4ac3-a9a7-1dc42c9da93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:30.132209 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc10483-b425-4ac3-a9a7-1dc42c9da93f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:30.132535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc10483-b425-4ac3-a9a7-1dc42c9da93f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:30.132807 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc10483-b425-4ac3-a9a7-1dc42c9da93f None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:30.133494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc10483-b425-4ac3-a9a7-1dc42c9da93f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:30.133788 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc10483-b425-4ac3-a9a7-1dc42c9da93f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:30.135279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc10483-b425-4ac3-a9a7-1dc42c9da93f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:30.158453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 281/559] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:30.163530 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc10483-b425-4ac3-a9a7-1dc42c9da93f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:30.166250 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:30.208971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed684af4-3960-4cef-b0d6-9fc93b9898e9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:30.254093 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e7a65-65c8-40f8-9726-2eb25b4c299f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:30.254696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e7a65-65c8-40f8-9726-2eb25b4c299f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:30.254975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e7a65-65c8-40f8-9726-2eb25b4c299f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:30.255253 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e7a65-65c8-40f8-9726-2eb25b4c299f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:30.305292 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc10483-b425-4ac3-a9a7-1dc42c9da93f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:30.339530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e7a65-65c8-40f8-9726-2eb25b4c299f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:30.340924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e7a65-65c8-40f8-9726-2eb25b4c299f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:30.342830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089b384-264f-4ca8-9137-045b7bd2fd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:30.343449 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089b384-264f-4ca8-9137-045b7bd2fd16 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:30.343720 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089b384-264f-4ca8-9137-045b7bd2fd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:30.343984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089b384-264f-4ca8-9137-045b7bd2fd16 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:30.352529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089b384-264f-4ca8-9137-045b7bd2fd16 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:30.353957 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089b384-264f-4ca8-9137-045b7bd2fd16 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:30.364862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-5089b384-264f-4ca8-9137-045b7bd2fd16 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:42:30.364862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:30.364862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:30.364862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:30.364862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:30.364862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:30.364862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:30.364862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:30.364862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:30.364862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:30.364862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:30.364862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:30.364862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:30.364862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:30.364862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:30.364862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:30.366426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:30.367443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:30.367443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:30.367443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:30.367443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:30.367443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:30.367443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:30.367443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:30.367443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:30.367443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:42:30.367443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:30.367443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 282/560] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 7 07:42:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:42:30.367443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089b384-264f-4ca8-9137-045b7bd2fd16 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:30.395234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:30.425884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:30.461421 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 279/561] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:30.462677 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6903ed-0718-4f92-b12f-516d915187f5 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:30.462956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6903ed-0718-4f92-b12f-516d915187f5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:30.463233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6903ed-0718-4f92-b12f-516d915187f5 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:30.468202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 280/562] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:30.468202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6903ed-0718-4f92-b12f-516d915187f5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:30.483351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089b384-264f-4ca8-9137-045b7bd2fd16 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:30.518040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c704783-12cf-4ad2-b904-b17eb9ebbf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:30.518670 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c704783-12cf-4ad2-b904-b17eb9ebbf14 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:30.518949 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c704783-12cf-4ad2-b904-b17eb9ebbf14 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:30.519219 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c704783-12cf-4ad2-b904-b17eb9ebbf14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:30.529270 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c704783-12cf-4ad2-b904-b17eb9ebbf14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:30.534850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-7c704783-12cf-4ad2-b904-b17eb9ebbf14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:30.534850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:30.534850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:30.534850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:30.534850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:30.534850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:30.534850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:30.534850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:30.534850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:30.534850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:30.534850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:30.534850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:30.534850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:30.534850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:30.534850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:30.534850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:30.535937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:30.535937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:30.535937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:30.535937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:30.535937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:30.535937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:30.535937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:30.535937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:30.535937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:30.535937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:30.535937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:30.535937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:30.535937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:30.535937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:30.535937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:30.535937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:30.536858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:30.536858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:30.536858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:30.536858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:30.536858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:30.536858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:30.536858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:30.536858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:30.536858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:30.536858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:30.536858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:30.536858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:30.536858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 283/563] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:30.536858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c704783-12cf-4ad2-b904-b17eb9ebbf14 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:30.571849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6903ed-0718-4f92-b12f-516d915187f5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:30.607611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b1b4d-b0da-4afa-af7f-efb563579592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:30.608144 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b1b4d-b0da-4afa-af7f-efb563579592 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:30.608423 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b1b4d-b0da-4afa-af7f-efb563579592 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:30.608687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b1b4d-b0da-4afa-af7f-efb563579592 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:30.609401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b1b4d-b0da-4afa-af7f-efb563579592 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:30.609696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b1b4d-b0da-4afa-af7f-efb563579592 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:30.611678 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b1b4d-b0da-4afa-af7f-efb563579592 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:30.632811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 281/564] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 7 07:42:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:30.633450 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:30.633620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b1b4d-b0da-4afa-af7f-efb563579592 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:30.652477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c704783-12cf-4ad2-b904-b17eb9ebbf14 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:30.660574 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:30.687255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1bc7e-4e10-465c-b884-1d9921d28ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:30.687877 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1bc7e-4e10-465c-b884-1d9921d28ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:30.688196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1bc7e-4e10-465c-b884-1d9921d28ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:30.689068 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1bc7e-4e10-465c-b884-1d9921d28ee0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:30.699686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1bc7e-4e10-465c-b884-1d9921d28ee0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:30.704901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-fea1bc7e-4e10-465c-b884-1d9921d28ee0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:30.704901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:30.704901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:30.704901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:30.704901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:30.704901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:30.704901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:30.704901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:30.704901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:30.704901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:30.704901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:30.704901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:30.704901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:30.704901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:30.704901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:30.704901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:30.705996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:30.705996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:30.705996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:30.705996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:30.705996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:30.705996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:30.705996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:30.705996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:30.705996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:30.705996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:30.705996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:30.705996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:30.705996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:30.705996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:30.705996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:30.705996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:30.706837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:30.706837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:30.706837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:30.706837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:30.706837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:30.706837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:30.706837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:30.706837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:30.706837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:30.706837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:30.706837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:30.706837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:30.706837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 284/565] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:30.707511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f3f33-bd15-49ab-9e71-8e5e0c92005d None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:30.707675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f3f33-bd15-49ab-9e71-8e5e0c92005d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:30.707947 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f3f33-bd15-49ab-9e71-8e5e0c92005d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:30.725872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-227f3f33-bd15-49ab-9e71-8e5e0c92005d None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:30.727055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-227f3f33-bd15-49ab-9e71-8e5e0c92005d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:30.731950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b1b4d-b0da-4afa-af7f-efb563579592 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:30.767151 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f90de2-b893-4b2e-ae7a-6496149a1ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:30.767783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f90de2-b893-4b2e-ae7a-6496149a1ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:30.768061 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f90de2-b893-4b2e-ae7a-6496149a1ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:30.768366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f90de2-b893-4b2e-ae7a-6496149a1ff1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:30.778199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f90de2-b893-4b2e-ae7a-6496149a1ff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:30.779651 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f90de2-b893-4b2e-ae7a-6496149a1ff1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:30.792567 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 285/566] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:30.793380 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f3f33-bd15-49ab-9e71-8e5e0c92005d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:30.794408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f0f90de2-b893-4b2e-ae7a-6496149a1ff1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:30.794408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:30.794408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:30.794408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:30.794408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:30.794408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:30.794408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:30.794408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:30.794408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:30.794408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:30.794408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:30.794408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:30.794408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:30.794408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:30.794408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:30.794408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:30.795306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:30.796624 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:30.796624 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:30.796624 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:30.796624 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 282/567] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:30.796624 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f90de2-b893-4b2e-ae7a-6496149a1ff1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:30.886325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:30.887208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:30.909433 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f90de2-b893-4b2e-ae7a-6496149a1ff1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:30.925819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f3f33-bd15-49ab-9e71-8e5e0c92005d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:30.946247 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba3128-5d53-48cd-82dc-a2fe97e96105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:30.946985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba3128-5d53-48cd-82dc-a2fe97e96105 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:30.947358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba3128-5d53-48cd-82dc-a2fe97e96105 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:30.947695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba3128-5d53-48cd-82dc-a2fe97e96105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:30.959096 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba3128-5d53-48cd-82dc-a2fe97e96105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:30.959910 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd932d-ed17-43a3-9711-ac2040859286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:30.960535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd932d-ed17-43a3-9711-ac2040859286 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:30.960649 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba3128-5d53-48cd-82dc-a2fe97e96105 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:30.960805 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd932d-ed17-43a3-9711-ac2040859286 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:30.961077 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd932d-ed17-43a3-9711-ac2040859286 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:30.961721 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd932d-ed17-43a3-9711-ac2040859286 None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:30.962018 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd932d-ed17-43a3-9711-ac2040859286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:30.963365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd932d-ed17-43a3-9711-ac2040859286 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:30.971905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 286/568] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:30.972703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd932d-ed17-43a3-9711-ac2040859286 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:30.976732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-42ba3128-5d53-48cd-82dc-a2fe97e96105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:30.976732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:30.976732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:30.976732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:30.976732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:30.976732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:30.976732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:30.976732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:30.976732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:30.976732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:30.976732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:30.976732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:30.976732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:30.976732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:30.976732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:30.976732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:30.977854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:30.978932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:30.978932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:30.978932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:30.980677 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 283/569] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:30.981579 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba3128-5d53-48cd-82dc-a2fe97e96105 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:31.076722 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd932d-ed17-43a3-9711-ac2040859286 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:31.093027 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba3128-5d53-48cd-82dc-a2fe97e96105 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:31.111115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5bd12-76e1-4933-8674-29459d414ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:31.111713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5bd12-76e1-4933-8674-29459d414ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:31.111990 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5bd12-76e1-4933-8674-29459d414ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:31.112311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5bd12-76e1-4933-8674-29459d414ab4 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:31.112957 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5bd12-76e1-4933-8674-29459d414ab4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:31.113268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5bd12-76e1-4933-8674-29459d414ab4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:31.114642 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5bd12-76e1-4933-8674-29459d414ab4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:31.116794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:31.123091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:31.123440 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 287/570] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:31.124371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5bd12-76e1-4933-8674-29459d414ab4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:31.129229 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e649e20-a2e5-42e1-b858-931031382bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:31.129837 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e649e20-a2e5-42e1-b858-931031382bfa None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:31.130116 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e649e20-a2e5-42e1-b858-931031382bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:31.130388 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e649e20-a2e5-42e1-b858-931031382bfa None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:31.131026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e649e20-a2e5-42e1-b858-931031382bfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:31.131326 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e649e20-a2e5-42e1-b858-931031382bfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:31.132710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e649e20-a2e5-42e1-b858-931031382bfa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:31.145415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 284/571] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:31.146135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e649e20-a2e5-42e1-b858-931031382bfa None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:31.229906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5bd12-76e1-4933-8674-29459d414ab4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:31.243754 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e649e20-a2e5-42e1-b858-931031382bfa None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:31.263919 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cb509-c2a2-4fb6-8145-f271d359c9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:31.264568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cb509-c2a2-4fb6-8145-f271d359c9aa None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:31.264848 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cb509-c2a2-4fb6-8145-f271d359c9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:31.265119 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cb509-c2a2-4fb6-8145-f271d359c9aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:31.275691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cb509-c2a2-4fb6-8145-f271d359c9aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:31.277012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981b0de-1224-4cc7-a9bf-b4feb52ca262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:31.277151 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cb509-c2a2-4fb6-8145-f271d359c9aa None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:31.277598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981b0de-1224-4cc7-a9bf-b4feb52ca262 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:31.277879 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981b0de-1224-4cc7-a9bf-b4feb52ca262 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:31.278143 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981b0de-1224-4cc7-a9bf-b4feb52ca262 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:31.278769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981b0de-1224-4cc7-a9bf-b4feb52ca262 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:31.279067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981b0de-1224-4cc7-a9bf-b4feb52ca262 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:31.280414 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981b0de-1224-4cc7-a9bf-b4feb52ca262 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:31.291202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-cf1cb509-c2a2-4fb6-8145-f271d359c9aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:31.291202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:31.291202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:31.291202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:31.291202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:31.291202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:31.291202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:31.291202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:31.291202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:31.291202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:31.291202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:31.291202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:31.291202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:31.291202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:31.291202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:31.291202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:31.292176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:31.293153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:31.293153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:31.293153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:31.293330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 285/572] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:31.293330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981b0de-1224-4cc7-a9bf-b4feb52ca262 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:31.293465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 288/573] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:31.293465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cb509-c2a2-4fb6-8145-f271d359c9aa None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:31.344501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:31.353295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:31.398599 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cb509-c2a2-4fb6-8145-f271d359c9aa None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:31.400503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981b0de-1224-4cc7-a9bf-b4feb52ca262 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:31.433870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc15b7-c46e-40eb-9b54-2887c8e173fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:31.434168 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc096375-84c3-41f9-a18e-15adc5ee4a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:31.434489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc15b7-c46e-40eb-9b54-2887c8e173fa None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:31.434760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc15b7-c46e-40eb-9b54-2887c8e173fa None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:31.434889 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc096375-84c3-41f9-a18e-15adc5ee4a7e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:31.435024 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc15b7-c46e-40eb-9b54-2887c8e173fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:31.435128 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc096375-84c3-41f9-a18e-15adc5ee4a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:31.435306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc096375-84c3-41f9-a18e-15adc5ee4a7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:31.446197 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc096375-84c3-41f9-a18e-15adc5ee4a7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:31.446420 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc15b7-c46e-40eb-9b54-2887c8e173fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:31.447852 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc15b7-c46e-40eb-9b54-2887c8e173fa None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:31.452251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-cc096375-84c3-41f9-a18e-15adc5ee4a7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:31.452251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:31.452251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:31.452251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:31.452251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:31.452251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:31.452251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:31.452251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:31.452251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:31.452251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:31.452251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:31.452251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:31.452251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:31.452251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:31.452251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:31.452251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:31.453157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:31.453157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:31.453157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:31.453157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:31.453157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:31.453157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:31.453157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:31.453157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:31.453157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:31.453157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:31.453157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:31.453157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:31.453157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:31.453157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:31.453157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:31.453157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:31.454013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:31.454013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:31.454013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:31.454013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:31.454013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:31.454013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:31.454013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:31.454013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:31.454013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:31.454013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:31.454013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:31.454013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:31.454013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 286/574] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:31.455013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc096375-84c3-41f9-a18e-15adc5ee4a7e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:31.463594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-c4dc15b7-c46e-40eb-9b54-2887c8e173fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:31.463594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:31.463594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:31.463594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:31.463594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:31.463594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:31.463594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:31.463594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:31.463594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:31.463594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:31.463594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:31.463594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:31.463594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:31.463594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:31.463594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:31.463594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:31.464684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:31.465554 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:31.465554 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:31.465554 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:31.465554 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 289/575] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:31.465554 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc15b7-c46e-40eb-9b54-2887c8e173fa None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:31.554312 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc096375-84c3-41f9-a18e-15adc5ee4a7e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:31.566959 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc15b7-c46e-40eb-9b54-2887c8e173fa None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:31.567392 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:31.568565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:31.589204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5982a3d-62cc-4e5f-bf9a-5bc77f614630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:31.589826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5982a3d-62cc-4e5f-bf9a-5bc77f614630 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:31.590099 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5982a3d-62cc-4e5f-bf9a-5bc77f614630 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:31.590367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5982a3d-62cc-4e5f-bf9a-5bc77f614630 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:31.597646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5982a3d-62cc-4e5f-bf9a-5bc77f614630 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:31.599392 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5982a3d-62cc-4e5f-bf9a-5bc77f614630 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:31.602229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a4de7-4af1-424d-91c4-179f53d9058b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:31.602838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a4de7-4af1-424d-91c4-179f53d9058b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:31.603111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a4de7-4af1-424d-91c4-179f53d9058b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:31.603377 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a4de7-4af1-424d-91c4-179f53d9058b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:31.609808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a4de7-4af1-424d-91c4-179f53d9058b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:31.610953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-b5982a3d-62cc-4e5f-bf9a-5bc77f614630 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:31.610953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:31.610953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:31.610953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:31.610953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:31.610953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:31.610953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:31.610953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:31.610953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:31.610953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:31.610953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:31.610953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:31.610953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:31.610953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:31.610953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:31.611902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:31.612831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:31.612831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:31.612831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:31.612831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:31.612831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:31.612831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:31.612831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:31.612831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:31.613260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a4de7-4af1-424d-91c4-179f53d9058b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:31.613330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 287/576] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:31.613330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5982a3d-62cc-4e5f-bf9a-5bc77f614630 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:31.624674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-751a4de7-4af1-424d-91c4-179f53d9058b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:31.624674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:31.624674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:31.624674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:31.624674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:31.624674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:31.624674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:31.624674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:31.624674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:31.624674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:31.624674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:31.624674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:31.624674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:31.624674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:31.624674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:31.625746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:31.627053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:31.627053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:31.627053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:31.627053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:31.627053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:31.627053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:31.627053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:31.627053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:31.630351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 290/577] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:31.631037 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a4de7-4af1-424d-91c4-179f53d9058b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:31.712869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5982a3d-62cc-4e5f-bf9a-5bc77f614630 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:31.729658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a4de7-4af1-424d-91c4-179f53d9058b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:31.745887 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf67968-3673-490c-a785-b77e35665466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:31.746482 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf67968-3673-490c-a785-b77e35665466 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:31.746760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf67968-3673-490c-a785-b77e35665466 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:31.747031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf67968-3673-490c-a785-b77e35665466 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:31.747669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf67968-3673-490c-a785-b77e35665466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:31.747971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf67968-3673-490c-a785-b77e35665466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:31.749365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf67968-3673-490c-a785-b77e35665466 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:31.761458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 288/578] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:31.762234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf67968-3673-490c-a785-b77e35665466 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:31.762801 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2df36-461e-431c-b177-d21919ccef83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:31.763404 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2df36-461e-431c-b177-d21919ccef83 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:31.763679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2df36-461e-431c-b177-d21919ccef83 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:31.763949 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2df36-461e-431c-b177-d21919ccef83 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:31.764656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2df36-461e-431c-b177-d21919ccef83 None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:31.764958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2df36-461e-431c-b177-d21919ccef83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:31.766320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2df36-461e-431c-b177-d21919ccef83 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:31.775420 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 291/579] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:31.776226 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2df36-461e-431c-b177-d21919ccef83 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:31.799655 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:31.801805 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:31.870319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf67968-3673-490c-a785-b77e35665466 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:31.876471 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2df36-461e-431c-b177-d21919ccef83 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:31.903103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184b260-95d8-4b21-809d-5b6ec81f83f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:31.903694 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184b260-95d8-4b21-809d-5b6ec81f83f7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:31.903968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184b260-95d8-4b21-809d-5b6ec81f83f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:31.904263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184b260-95d8-4b21-809d-5b6ec81f83f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:31.904886 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184b260-95d8-4b21-809d-5b6ec81f83f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:31.905181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184b260-95d8-4b21-809d-5b6ec81f83f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:31.906516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184b260-95d8-4b21-809d-5b6ec81f83f7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:31.910476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb9714-ff4e-47e8-84d8-e108c5cf2fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:31.911070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb9714-ff4e-47e8-84d8-e108c5cf2fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:31.911358 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb9714-ff4e-47e8-84d8-e108c5cf2fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:31.911659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb9714-ff4e-47e8-84d8-e108c5cf2fb0 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:31.912423 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb9714-ff4e-47e8-84d8-e108c5cf2fb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:31.912758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb9714-ff4e-47e8-84d8-e108c5cf2fb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:31.914337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb9714-ff4e-47e8-84d8-e108c5cf2fb0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:31.918049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 289/580] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:31.918827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184b260-95d8-4b21-809d-5b6ec81f83f7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:31.927911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 292/581] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:31.928739 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb9714-ff4e-47e8-84d8-e108c5cf2fb0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:32.012146 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:32.013561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:32.023055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184b260-95d8-4b21-809d-5b6ec81f83f7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:32.030373 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb9714-ff4e-47e8-84d8-e108c5cf2fb0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:32.055992 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3cefe-be26-4be1-999b-a6de9d2463d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:32.056615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3cefe-be26-4be1-999b-a6de9d2463d3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:32.056892 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3cefe-be26-4be1-999b-a6de9d2463d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:32.057159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3cefe-be26-4be1-999b-a6de9d2463d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:32.057834 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f3cefe-be26-4be1-999b-a6de9d2463d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:32.058138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f3cefe-be26-4be1-999b-a6de9d2463d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:32.059545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f3cefe-be26-4be1-999b-a6de9d2463d3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:32.063872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006a286-e880-4f1c-bc40-809f7c79bf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:32.064579 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006a286-e880-4f1c-bc40-809f7c79bf28 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:32.064887 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006a286-e880-4f1c-bc40-809f7c79bf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:32.065157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006a286-e880-4f1c-bc40-809f7c79bf28 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:32.071339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 290/582] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:32.072230 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3cefe-be26-4be1-999b-a6de9d2463d3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:32.077372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0006a286-e880-4f1c-bc40-809f7c79bf28 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:32.078802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0006a286-e880-4f1c-bc40-809f7c79bf28 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:32.091283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-0006a286-e880-4f1c-bc40-809f7c79bf28 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 07:42:32.091283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:32.091283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:32.091283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:32.091283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:32.091283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:32.091283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:32.091283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:32.091283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:32.091283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:32.091283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:32.091283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:32.091283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:32.091283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:32.091283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:32.091283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:32.097847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:32.099139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:32.099139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:32.099139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:32.099139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:32.099139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:32.099139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:32.099139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:32.099139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:32.099139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 07:42:32.099139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:32.099139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 293/583] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:32.099139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006a286-e880-4f1c-bc40-809f7c79bf28 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:32.178751 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3cefe-be26-4be1-999b-a6de9d2463d3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:32.192511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006a286-e880-4f1c-bc40-809f7c79bf28 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:32.212467 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500fb16-9702-4850-ac58-231a7c0d5624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:32.213088 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500fb16-9702-4850-ac58-231a7c0d5624 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:32.213365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500fb16-9702-4850-ac58-231a7c0d5624 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:32.213635 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500fb16-9702-4850-ac58-231a7c0d5624 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:32.224557 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a500fb16-9702-4850-ac58-231a7c0d5624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:32.225984 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a500fb16-9702-4850-ac58-231a7c0d5624 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:32.226782 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f967332-4658-4928-b68c-b9bb2e41d7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:32.227393 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f967332-4658-4928-b68c-b9bb2e41d7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:32.227678 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f967332-4658-4928-b68c-b9bb2e41d7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:32.227943 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f967332-4658-4928-b68c-b9bb2e41d7f5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:32.235283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f967332-4658-4928-b68c-b9bb2e41d7f5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:32.236822 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f967332-4658-4928-b68c-b9bb2e41d7f5 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:32.241432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-a500fb16-9702-4850-ac58-231a7c0d5624 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:32.241432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:32.241432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:32.241432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:32.241432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:32.241432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:32.241432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:32.241432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:32.241432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:32.241432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:32.241432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:32.241432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:32.241432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:32.241432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:32.241432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:32.241432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:32.247119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:32.248331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:32.248331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:32.248331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:32.248331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 291/584] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:32.248331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500fb16-9702-4850-ac58-231a7c0d5624 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:32.248331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:32.248733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-2f967332-4658-4928-b68c-b9bb2e41d7f5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 07:42:32.248733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:32.248733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:32.248733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:32.248733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:32.248733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:32.248733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:32.248733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:32.248733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:32.248733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:32.248733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:32.248733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:32.248733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:32.248733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:32.248733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:32.248733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:32.250682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:32.252837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:32.252837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:32.252837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:32.252837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:32.252837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:32.252837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:32.252837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:32.252837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:32.252837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 07:42:32.252837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:32.252837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 294/585] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:32.252837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f967332-4658-4928-b68c-b9bb2e41d7f5 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:32.257673 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:32.348314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500fb16-9702-4850-ac58-231a7c0d5624 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:32.353990 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f967332-4658-4928-b68c-b9bb2e41d7f5 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:32.381977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5058a0-1d32-4702-af79-a79b5f3efb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:32.382891 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5058a0-1d32-4702-af79-a79b5f3efb0e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:32.383176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5058a0-1d32-4702-af79-a79b5f3efb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:32.383456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5058a0-1d32-4702-af79-a79b5f3efb0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:32.384246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5058a0-1d32-4702-af79-a79b5f3efb0e None admin] RBAC: Adding query filter params (role.id=e7960e6d957f4a1fa6547565db026f24, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=3ff3bf8da732489b8b3a8fbe5aa22f21) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:32.384560 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5058a0-1d32-4702-af79-a79b5f3efb0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:32.385976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5058a0-1d32-4702-af79-a79b5f3efb0e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:32.387110 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8924e38-cd99-43ed-89ae-e657d72caa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:32.387761 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8924e38-cd99-43ed-89ae-e657d72caa93 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:32.388042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8924e38-cd99-43ed-89ae-e657d72caa93 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:32.388342 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8924e38-cd99-43ed-89ae-e657d72caa93 None admin] PATH_INFO: `/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/81d5689e9a344bafb74ca423ffd2b42a/roles/e7960e6d957f4a1fa6547565db026f24` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:32.391334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 292/586] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:42:32 2022] GET /identity/v3/role_assignments?user.id=3ff3bf8da732489b8b3a8fbe5aa22f21&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=e7960e6d957f4a1fa6547565db026f24 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:32.392980 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5058a0-1d32-4702-af79-a79b5f3efb0e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:32.407630 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8924e38-cd99-43ed-89ae-e657d72caa93 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a949b959e584474aafb4a05ebbabcc4, user_id=81d5689e9a344bafb74ca423ffd2b42a, role_id=e7960e6d957f4a1fa6547565db026f24)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:32.409317 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8924e38-cd99-43ed-89ae-e657d72caa93 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:32.423912 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 295/587] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:42:32 2022] PUT /identity/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/81d5689e9a344bafb74ca423ffd2b42a/roles/e7960e6d957f4a1fa6547565db026f24 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:42:32.424709 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8924e38-cd99-43ed-89ae-e657d72caa93 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:32.504286 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:32.508784 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:32.564783 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8924e38-cd99-43ed-89ae-e657d72caa93 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:32.567026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5058a0-1d32-4702-af79-a79b5f3efb0e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:32.611923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeca8e4-9336-44a7-b411-6e4d20823e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:32.612465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeca8e4-9336-44a7-b411-6e4d20823e7d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:32.612745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeca8e4-9336-44a7-b411-6e4d20823e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:32.613012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeca8e4-9336-44a7-b411-6e4d20823e7d None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:32.613647 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeca8e4-9336-44a7-b411-6e4d20823e7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:32.613943 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeca8e4-9336-44a7-b411-6e4d20823e7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:32.614553 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8518911-d4d1-4e94-b434-2f984a7675fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:32.615386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8518911-d4d1-4e94-b434-2f984a7675fc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:32.615491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeca8e4-9336-44a7-b411-6e4d20823e7d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:32.615962 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8518911-d4d1-4e94-b434-2f984a7675fc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:32.616449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8518911-d4d1-4e94-b434-2f984a7675fc None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:32.617378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8518911-d4d1-4e94-b434-2f984a7675fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:32.617903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8518911-d4d1-4e94-b434-2f984a7675fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:32.619516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8518911-d4d1-4e94-b434-2f984a7675fc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:32.629502 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 296/588] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:32.630356 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeca8e4-9336-44a7-b411-6e4d20823e7d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:32.640133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 293/589] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Sep 7 07:42:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:32.641336 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8518911-d4d1-4e94-b434-2f984a7675fc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:32.742250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeca8e4-9336-44a7-b411-6e4d20823e7d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:32.745324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8518911-d4d1-4e94-b434-2f984a7675fc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:32.757490 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:32.773023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:32.782500 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38d178-fb4c-45a0-b44c-d3229ea0842a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:32.783031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2cd35-fc90-4554-bd84-2f07b60c6c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:32.783681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38d178-fb4c-45a0-b44c-d3229ea0842a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:32.783833 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2cd35-fc90-4554-bd84-2f07b60c6c7d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:32.784237 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2cd35-fc90-4554-bd84-2f07b60c6c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:32.784455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38d178-fb4c-45a0-b44c-d3229ea0842a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:32.784689 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2cd35-fc90-4554-bd84-2f07b60c6c7d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:32.785035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38d178-fb4c-45a0-b44c-d3229ea0842a None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:32.785921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38d178-fb4c-45a0-b44c-d3229ea0842a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:32.786363 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38d178-fb4c-45a0-b44c-d3229ea0842a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:32.788158 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38d178-fb4c-45a0-b44c-d3229ea0842a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:32.792897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2cd35-fc90-4554-bd84-2f07b60c6c7d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:32.794437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2cd35-fc90-4554-bd84-2f07b60c6c7d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:32.807207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-11d2cd35-fc90-4554-bd84-2f07b60c6c7d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 07:42:32.807207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:32.807207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:32.807207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:32.807207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:32.807207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:32.807207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:32.807207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:32.807207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:32.807207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:32.807207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:32.807207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:32.807207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:32.807207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:32.807207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:32.807207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:32.808405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:32.809523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:32.809523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:32.809523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:32.809523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:32.809523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:32.809523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:32.809523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:32.809523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:32.809523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 07:42:32.809523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:32.811148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 297/591] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Sep 7 07:42:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:32.811269 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 294/591] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:32.820138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2cd35-fc90-4554-bd84-2f07b60c6c7d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:32.820307 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38d178-fb4c-45a0-b44c-d3229ea0842a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:32.925443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2cd35-fc90-4554-bd84-2f07b60c6c7d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:32.948464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38d178-fb4c-45a0-b44c-d3229ea0842a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:32.972358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641e644-1597-495a-a666-12f6f7d1c430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:32.973916 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641e644-1597-495a-a666-12f6f7d1c430 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:32.974440 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641e644-1597-495a-a666-12f6f7d1c430 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:32.974849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641e644-1597-495a-a666-12f6f7d1c430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:32.990112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4641e644-1597-495a-a666-12f6f7d1c430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:32.992474 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4641e644-1597-495a-a666-12f6f7d1c430 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:32.997451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee8ed4-96f7-4398-a0d2-7cdd27335e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:32.998230 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee8ed4-96f7-4398-a0d2-7cdd27335e1f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:32.998609 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee8ed4-96f7-4398-a0d2-7cdd27335e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:32.998981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee8ed4-96f7-4398-a0d2-7cdd27335e1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:33.009970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-4641e644-1597-495a-a666-12f6f7d1c430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:33.009970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:33.009970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:33.009970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:33.009970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:33.009970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:33.009970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:33.009970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:33.009970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:33.009970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:33.009970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:33.009970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:33.009970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:33.009970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:33.009970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:33.009970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:33.011055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:33.012139 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:33.012139 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:33.012139 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:33.019735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 295/592] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:33.019735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641e644-1597-495a-a666-12f6f7d1c430 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:33.020490 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee8ed4-96f7-4398-a0d2-7cdd27335e1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:33.021958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee8ed4-96f7-4398-a0d2-7cdd27335e1f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:33.043565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-09ee8ed4-96f7-4398-a0d2-7cdd27335e1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:33.043565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:33.043565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:33.043565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:33.043565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:33.043565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:33.043565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:33.043565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:33.043565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:33.043565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:33.043565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:33.043565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:33.043565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:33.043565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:33.043565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:33.043565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:33.044570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:33.045437 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:33.045437 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:33.045437 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:33.047617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 298/593] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:33.049390 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee8ed4-96f7-4398-a0d2-7cdd27335e1f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:33.064572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:33.088435 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:33.129219 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641e644-1597-495a-a666-12f6f7d1c430 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:33.166085 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f251f1-dc51-4bc2-bc9d-eef8933a636a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:33.166694 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f251f1-dc51-4bc2-bc9d-eef8933a636a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:33.166972 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f251f1-dc51-4bc2-bc9d-eef8933a636a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:33.167244 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f251f1-dc51-4bc2-bc9d-eef8933a636a None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:33.168038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f251f1-dc51-4bc2-bc9d-eef8933a636a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:33.168380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f251f1-dc51-4bc2-bc9d-eef8933a636a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:33.169926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f251f1-dc51-4bc2-bc9d-eef8933a636a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:33.192404 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 296/594] 10.222.0.26 () {56 vars in 1139 bytes} [Wed Sep 7 07:42:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:33.193177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f251f1-dc51-4bc2-bc9d-eef8933a636a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:33.200172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee8ed4-96f7-4398-a0d2-7cdd27335e1f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:33.238231 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05a6f3-8bf8-4226-bc4b-7afe9e165b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:33.238904 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05a6f3-8bf8-4226-bc4b-7afe9e165b26 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:33.239187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05a6f3-8bf8-4226-bc4b-7afe9e165b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:33.239466 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05a6f3-8bf8-4226-bc4b-7afe9e165b26 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:33.240119 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05a6f3-8bf8-4226-bc4b-7afe9e165b26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:33.240427 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05a6f3-8bf8-4226-bc4b-7afe9e165b26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:33.241808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05a6f3-8bf8-4226-bc4b-7afe9e165b26 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:33.255858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 299/595] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:33.256846 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05a6f3-8bf8-4226-bc4b-7afe9e165b26 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:33.320709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:33.330583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f251f1-dc51-4bc2-bc9d-eef8933a636a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:33.354328 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:33.381904 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05a6f3-8bf8-4226-bc4b-7afe9e165b26 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:33.384556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5ded8-9c92-43fc-8e53-88d83cee6d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:33.386170 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5ded8-9c92-43fc-8e53-88d83cee6d2e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:33.386448 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5ded8-9c92-43fc-8e53-88d83cee6d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:33.386724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5ded8-9c92-43fc-8e53-88d83cee6d2e None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:33.387354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5ded8-9c92-43fc-8e53-88d83cee6d2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:33.392108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5ded8-9c92-43fc-8e53-88d83cee6d2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:33.392108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5ded8-9c92-43fc-8e53-88d83cee6d2e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:33.408177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 297/596] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:33.409330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5ded8-9c92-43fc-8e53-88d83cee6d2e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:33.430870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7717401-ccd2-4f23-a643-b580babb2942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:33.431683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7717401-ccd2-4f23-a643-b580babb2942 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:33.432065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7717401-ccd2-4f23-a643-b580babb2942 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:33.432371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7717401-ccd2-4f23-a643-b580babb2942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:33.444755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7717401-ccd2-4f23-a643-b580babb2942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:33.446388 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7717401-ccd2-4f23-a643-b580babb2942 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:33.463153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-a7717401-ccd2-4f23-a643-b580babb2942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:33.463153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:33.463153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:33.463153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:33.463153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:33.463153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:33.463153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:33.463153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:33.463153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:33.463153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:33.463153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:33.463153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:33.463153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:33.463153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:33.463153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:33.463153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:33.464162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:33.465062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:33.465062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:33.465062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:33.468215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 300/597] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:33.469124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7717401-ccd2-4f23-a643-b580babb2942 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:33.532960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5ded8-9c92-43fc-8e53-88d83cee6d2e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:33.577311 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e9d32-2aa0-4642-8747-e146d9ea3886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:33.578246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e9d32-2aa0-4642-8747-e146d9ea3886 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:33.578778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e9d32-2aa0-4642-8747-e146d9ea3886 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:33.579589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e9d32-2aa0-4642-8747-e146d9ea3886 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:33.581390 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7717401-ccd2-4f23-a643-b580babb2942 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:33.587578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e9d32-2aa0-4642-8747-e146d9ea3886 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:33.589473 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e9d32-2aa0-4642-8747-e146d9ea3886 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:33.604271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d06e9d32-2aa0-4642-8747-e146d9ea3886 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:33.604271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:33.604271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:33.604271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:33.604271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:33.604271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:33.604271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:33.604271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:33.604271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:33.604271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:33.604271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:33.604271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:33.604271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:33.604271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:33.604271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:33.606183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:33.607383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:33.607383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:33.607383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:33.607383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:33.607383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:33.607383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:33.607383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:33.607383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:33.607383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 298/598] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:33.607383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e9d32-2aa0-4642-8747-e146d9ea3886 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:33.627282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37d608-83f9-44f3-b68f-95c859408920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:33.628355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:33.628960 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37d608-83f9-44f3-b68f-95c859408920 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:33.629449 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37d608-83f9-44f3-b68f-95c859408920 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:33.629876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37d608-83f9-44f3-b68f-95c859408920 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:33.633569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:33.641229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37d608-83f9-44f3-b68f-95c859408920 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:33.643909 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37d608-83f9-44f3-b68f-95c859408920 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:33.659483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-cc37d608-83f9-44f3-b68f-95c859408920 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:33.659483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:33.659483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:33.659483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:33.659483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:33.659483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:33.659483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:33.659483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:33.659483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:33.659483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:33.659483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:33.659483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:33.659483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:33.659483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:33.659483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:33.660570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:33.661638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:33.661638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:33.661638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:33.661638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:33.661638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:33.661638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:33.661638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:33.661638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:33.668719 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 301/599] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:33.669108 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37d608-83f9-44f3-b68f-95c859408920 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:33.738535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e9d32-2aa0-4642-8747-e146d9ea3886 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:33.787060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96afc2a-9cf1-4154-ad0b-0f343bb39b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:33.787666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96afc2a-9cf1-4154-ad0b-0f343bb39b47 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:33.787947 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96afc2a-9cf1-4154-ad0b-0f343bb39b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:33.788256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96afc2a-9cf1-4154-ad0b-0f343bb39b47 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:33.788883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37d608-83f9-44f3-b68f-95c859408920 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:33.789001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96afc2a-9cf1-4154-ad0b-0f343bb39b47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:33.789195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96afc2a-9cf1-4154-ad0b-0f343bb39b47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:33.790550 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96afc2a-9cf1-4154-ad0b-0f343bb39b47 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:33.812165 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 299/600] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:33.815208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96afc2a-9cf1-4154-ad0b-0f343bb39b47 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:33.830208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eddeff4-7da3-4e50-817e-8cb207cf2a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:33.831140 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddeff4-7da3-4e50-817e-8cb207cf2a25 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:33.831646 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddeff4-7da3-4e50-817e-8cb207cf2a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:33.832169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddeff4-7da3-4e50-817e-8cb207cf2a25 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:33.833067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eddeff4-7da3-4e50-817e-8cb207cf2a25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:33.833586 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eddeff4-7da3-4e50-817e-8cb207cf2a25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:33.835279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eddeff4-7da3-4e50-817e-8cb207cf2a25 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:33.849001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 302/601] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:33.850070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eddeff4-7da3-4e50-817e-8cb207cf2a25 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:33.913467 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:33.916354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:33.946124 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96afc2a-9cf1-4154-ad0b-0f343bb39b47 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:33.966637 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eddeff4-7da3-4e50-817e-8cb207cf2a25 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:33.993962 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1cc75-d282-471d-ba11-a43f5fddd749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:33.995028 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1cc75-d282-471d-ba11-a43f5fddd749 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:33.995860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1cc75-d282-471d-ba11-a43f5fddd749 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:33.996317 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1cc75-d282-471d-ba11-a43f5fddd749 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:33.997458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1cc75-d282-471d-ba11-a43f5fddd749 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:33.998160 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1cc75-d282-471d-ba11-a43f5fddd749 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:34.000429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1cc75-d282-471d-ba11-a43f5fddd749 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:34.004058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4280f-f3d3-4e79-8855-a9ca70fc8976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:34.004823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4280f-f3d3-4e79-8855-a9ca70fc8976 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:34.005205 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4280f-f3d3-4e79-8855-a9ca70fc8976 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:34.005581 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4280f-f3d3-4e79-8855-a9ca70fc8976 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:34.012991 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4280f-f3d3-4e79-8855-a9ca70fc8976 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:34.014533 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 300/602] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:34.015041 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4280f-f3d3-4e79-8855-a9ca70fc8976 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:34.016006 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1cc75-d282-471d-ba11-a43f5fddd749 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:34.032978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-d0a4280f-f3d3-4e79-8855-a9ca70fc8976 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:34.032978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:34.032978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:34.032978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:34.032978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:34.032978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:34.032978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:34.032978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:34.032978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:34.032978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:34.032978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:34.032978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:34.032978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:34.032978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:34.032978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:34.034147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:34.035206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:34.035206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:34.035206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:34.035206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:34.035206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:34.035206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:34.035206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:34.035206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:34.041613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 303/603] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:34.041613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4280f-f3d3-4e79-8855-a9ca70fc8976 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:34.145750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1cc75-d282-471d-ba11-a43f5fddd749 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:34.174829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4280f-f3d3-4e79-8855-a9ca70fc8976 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:34.181527 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7a97a-dd92-470c-8bdb-a6116d26347f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:34.182311 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7a97a-dd92-470c-8bdb-a6116d26347f None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:34.182695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7a97a-dd92-470c-8bdb-a6116d26347f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:34.183071 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7a97a-dd92-470c-8bdb-a6116d26347f None admin] PATH_INFO: `/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/9b0652c89c9945e2866512c9da4793ba/roles/e7960e6d957f4a1fa6547565db026f24` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:34.187426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:34.199939 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:34.202145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7a97a-dd92-470c-8bdb-a6116d26347f None admin] RBAC: Authorizing `identity:create_grant(project_id=9a949b959e584474aafb4a05ebbabcc4, user_id=9b0652c89c9945e2866512c9da4793ba, role_id=e7960e6d957f4a1fa6547565db026f24)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:34.203748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7a97a-dd92-470c-8bdb-a6116d26347f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:34.222401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 301/604] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:42:34 2022] PUT /identity/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/9b0652c89c9945e2866512c9da4793ba/roles/e7960e6d957f4a1fa6547565db026f24 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:42:34.223206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7a97a-dd92-470c-8bdb-a6116d26347f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:34.226253 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0be28-9e5d-4098-b1ba-d8c13d6c9c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:34.227332 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0be28-9e5d-4098-b1ba-d8c13d6c9c70 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:34.227884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0be28-9e5d-4098-b1ba-d8c13d6c9c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:34.228423 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0be28-9e5d-4098-b1ba-d8c13d6c9c70 None admin] PATH_INFO: `/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/92699b512c064b05948afe597650fdb6/roles/e7960e6d957f4a1fa6547565db026f24` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:34.246938 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0be28-9e5d-4098-b1ba-d8c13d6c9c70 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a949b959e584474aafb4a05ebbabcc4, user_id=92699b512c064b05948afe597650fdb6, role_id=e7960e6d957f4a1fa6547565db026f24)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:34.249160 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0be28-9e5d-4098-b1ba-d8c13d6c9c70 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:34.267610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 304/605] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:42:34 2022] PUT /identity/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/92699b512c064b05948afe597650fdb6/roles/e7960e6d957f4a1fa6547565db026f24 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:42:34.370290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7a97a-dd92-470c-8bdb-a6116d26347f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:34.409962 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ec556-df04-47e9-af9c-3e3a42cb7562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:34.410590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ec556-df04-47e9-af9c-3e3a42cb7562 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:34.410865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ec556-df04-47e9-af9c-3e3a42cb7562 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:34.411135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ec556-df04-47e9-af9c-3e3a42cb7562 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:34.411846 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ec556-df04-47e9-af9c-3e3a42cb7562 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:34.412182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ec556-df04-47e9-af9c-3e3a42cb7562 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:34.415265 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ec556-df04-47e9-af9c-3e3a42cb7562 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:34.427886 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 302/606] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:34.433615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0be28-9e5d-4098-b1ba-d8c13d6c9c70 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:34.554426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0be28-9e5d-4098-b1ba-d8c13d6c9c70 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:34.599849 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0088fd9-f907-4cc8-8fd1-c8e4b358b024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:34.600818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0088fd9-f907-4cc8-8fd1-c8e4b358b024 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:34.601263 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0088fd9-f907-4cc8-8fd1-c8e4b358b024 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:34.601685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0088fd9-f907-4cc8-8fd1-c8e4b358b024 None admin] PATH_INFO: `/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/330a38822d33439191773e9a8b4cbe04/roles/e7960e6d957f4a1fa6547565db026f24` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:34.618859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0088fd9-f907-4cc8-8fd1-c8e4b358b024 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a949b959e584474aafb4a05ebbabcc4, user_id=330a38822d33439191773e9a8b4cbe04, role_id=e7960e6d957f4a1fa6547565db026f24)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:34.620659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0088fd9-f907-4cc8-8fd1-c8e4b358b024 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:34.639445 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 305/607] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:42:34 2022] PUT /identity/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/330a38822d33439191773e9a8b4cbe04/roles/e7960e6d957f4a1fa6547565db026f24 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:42:34.765034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361083c-f936-4f42-a651-f782445dd9af None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:34.765389 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361083c-f936-4f42-a651-f782445dd9af None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:34.765583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361083c-f936-4f42-a651-f782445dd9af None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:34.767304 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 303/608] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:34.772111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33362c-6448-4d19-97cb-83fd42f4877b None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:34.772409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33362c-6448-4d19-97cb-83fd42f4877b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:34.772647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfe397-29fd-4828-b5c6-ae803eaad634 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:34.772750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33362c-6448-4d19-97cb-83fd42f4877b None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:34.772934 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfe397-29fd-4828-b5c6-ae803eaad634 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:34.773202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfe397-29fd-4828-b5c6-ae803eaad634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:34.774096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 306/609] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:34.782591 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f887c48-b4f4-4371-a07a-467ad0cf7319 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:34.782895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f887c48-b4f4-4371-a07a-467ad0cf7319 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:34.783089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f887c48-b4f4-4371-a07a-467ad0cf7319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:34.797200 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b6bfe397-29fd-4828-b5c6-ae803eaad634 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:34.799345 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b6bfe397-29fd-4828-b5c6-ae803eaad634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:34.817992 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-0f887c48-b4f4-4371-a07a-467ad0cf7319 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:34.819150 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0f887c48-b4f4-4371-a07a-467ad0cf7319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:34.835866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:34.858661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:34.891900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 304/610] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:34.906105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 307/611] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:35.228020 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfe397-29fd-4828-b5c6-ae803eaad634 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:35.250250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f887c48-b4f4-4371-a07a-467ad0cf7319 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:35.343450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfe397-29fd-4828-b5c6-ae803eaad634 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:35.389399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae168f-e4c2-4e15-9ae2-40432c15bd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:35.390390 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae168f-e4c2-4e15-9ae2-40432c15bd5b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:35.390890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae168f-e4c2-4e15-9ae2-40432c15bd5b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:35.391423 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae168f-e4c2-4e15-9ae2-40432c15bd5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:35.407348 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae168f-e4c2-4e15-9ae2-40432c15bd5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:35.416294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d2ae168f-e4c2-4e15-9ae2-40432c15bd5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:35.416294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:35.416294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:35.416294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:35.416294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:35.416294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:35.416294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:35.416294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:35.416294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:35.416294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:35.416294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:35.416294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:35.416294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:35.416294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:35.416294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:35.416294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:35.417300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:35.417300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:35.417300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:35.417300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:35.417300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:35.417300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:35.417300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:35.417300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:35.417300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:35.417300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:35.417300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:35.417300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:35.417300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:35.417300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:35.417300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:35.417300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:35.418172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:35.418172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:35.418172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:35.418172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:35.418172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:35.418172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:35.418172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:35.418172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:35.418172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:35.418172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:35.418172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:35.418172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:35.418172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 305/612] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:35.430961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae168f-e4c2-4e15-9ae2-40432c15bd5b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:35.492527 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f887c48-b4f4-4371-a07a-467ad0cf7319 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:35.512469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:35.572809 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495045c-7adc-4ff5-a7a4-705f913e5cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:35.573442 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495045c-7adc-4ff5-a7a4-705f913e5cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:35.573715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495045c-7adc-4ff5-a7a4-705f913e5cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:35.573980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495045c-7adc-4ff5-a7a4-705f913e5cb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:35.592054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495045c-7adc-4ff5-a7a4-705f913e5cb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:35.597006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-6495045c-7adc-4ff5-a7a4-705f913e5cb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:35.597006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:35.597006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:35.597006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:35.597006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:35.597006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:35.597006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:35.597006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:35.597006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:35.597006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:35.597006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:35.597006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:35.597006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:35.597006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:35.597006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:35.597006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:35.598118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:35.598118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:35.598118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:35.598118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:35.598118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:35.598118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:35.598118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:35.598118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:35.598118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:35.598118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:35.598118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:35.598118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:35.598118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:35.598118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:35.598118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:35.598118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:35.599143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:35.599143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:35.599143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:35.599143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:35.599143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:35.599143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:35.599143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:35.599143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:35.599143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:35.599143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:35.599143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:35.599143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:35.603874 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae168f-e4c2-4e15-9ae2-40432c15bd5b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:35.606593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 308/613] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:35.613336 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495045c-7adc-4ff5-a7a4-705f913e5cb4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:35.628300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:35.671538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bd6a1-d2e0-45b6-9bf7-15e99aad32d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:35.672170 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bd6a1-d2e0-45b6-9bf7-15e99aad32d8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:35.672498 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bd6a1-d2e0-45b6-9bf7-15e99aad32d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:35.673296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bd6a1-d2e0-45b6-9bf7-15e99aad32d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:35.673963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9bd6a1-d2e0-45b6-9bf7-15e99aad32d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:35.674264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9bd6a1-d2e0-45b6-9bf7-15e99aad32d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:35.675641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9bd6a1-d2e0-45b6-9bf7-15e99aad32d8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:35.692228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 306/614] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:35.697862 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bd6a1-d2e0-45b6-9bf7-15e99aad32d8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:35.740516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495045c-7adc-4ff5-a7a4-705f913e5cb4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:35.778108 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418db724-58c3-44e4-a66f-f54dbd0bf26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:35.778633 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418db724-58c3-44e4-a66f-f54dbd0bf26e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:35.778913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418db724-58c3-44e4-a66f-f54dbd0bf26e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:35.779186 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418db724-58c3-44e4-a66f-f54dbd0bf26e None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:35.779840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418db724-58c3-44e4-a66f-f54dbd0bf26e None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:35.780169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418db724-58c3-44e4-a66f-f54dbd0bf26e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:35.781542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418db724-58c3-44e4-a66f-f54dbd0bf26e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:35.790855 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 309/615] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:35.800124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418db724-58c3-44e4-a66f-f54dbd0bf26e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:35.887516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:35.918463 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:35.925496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bd6a1-d2e0-45b6-9bf7-15e99aad32d8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:35.932527 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418db724-58c3-44e4-a66f-f54dbd0bf26e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:35.974660 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81e5a5-676b-4520-a8f3-19f98c0c37d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:35.975290 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81e5a5-676b-4520-a8f3-19f98c0c37d7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:35.975576 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81e5a5-676b-4520-a8f3-19f98c0c37d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:35.975846 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81e5a5-676b-4520-a8f3-19f98c0c37d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:35.988784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81e5a5-676b-4520-a8f3-19f98c0c37d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:35.990222 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81e5a5-676b-4520-a8f3-19f98c0c37d7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:36.007593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-2d81e5a5-676b-4520-a8f3-19f98c0c37d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:36.007593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:36.007593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:36.007593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:36.007593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:36.007593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:36.007593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:36.007593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:36.007593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:36.007593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:36.007593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:36.007593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:36.007593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:36.007593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:36.007593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:36.007593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:36.008603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:36.009474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:36.009474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:36.009474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:36.009474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 310/616] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:36.017847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81e5a5-676b-4520-a8f3-19f98c0c37d7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:36.019542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34275ee4-5604-4664-ae21-beb13fad3d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:36.020223 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34275ee4-5604-4664-ae21-beb13fad3d58 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:36.020874 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34275ee4-5604-4664-ae21-beb13fad3d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:36.021154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34275ee4-5604-4664-ae21-beb13fad3d58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:36.036002 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34275ee4-5604-4664-ae21-beb13fad3d58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:36.038026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34275ee4-5604-4664-ae21-beb13fad3d58 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:36.058581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-34275ee4-5604-4664-ae21-beb13fad3d58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:36.058581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:36.058581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:36.058581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:36.058581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:36.058581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:36.058581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:36.058581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:36.058581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:36.058581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:36.058581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:36.058581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:36.058581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:36.058581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:36.058581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:36.058581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:36.059720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:36.060833 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:36.060833 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:36.060833 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:36.064639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 307/617] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:36.070942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34275ee4-5604-4664-ae21-beb13fad3d58 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:36.195091 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81e5a5-676b-4520-a8f3-19f98c0c37d7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:36.217123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:36.234186 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ac054-6f76-4b6f-bcf1-f4ee30bbd9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:36.234795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ac054-6f76-4b6f-bcf1-f4ee30bbd9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:36.235073 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ac054-6f76-4b6f-bcf1-f4ee30bbd9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:36.235380 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ac054-6f76-4b6f-bcf1-f4ee30bbd9b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:36.236228 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ac054-6f76-4b6f-bcf1-f4ee30bbd9b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:36.236537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ac054-6f76-4b6f-bcf1-f4ee30bbd9b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:36.238040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ac054-6f76-4b6f-bcf1-f4ee30bbd9b2 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:36.256109 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 311/618] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:36.257343 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ac054-6f76-4b6f-bcf1-f4ee30bbd9b2 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:36.292599 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34275ee4-5604-4664-ae21-beb13fad3d58 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:36.341773 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a83f28-ef6d-450b-bd2e-dddf219c113f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:36.342371 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a83f28-ef6d-450b-bd2e-dddf219c113f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:36.342649 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a83f28-ef6d-450b-bd2e-dddf219c113f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:36.342919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a83f28-ef6d-450b-bd2e-dddf219c113f None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:36.343738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a83f28-ef6d-450b-bd2e-dddf219c113f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:36.344045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a83f28-ef6d-450b-bd2e-dddf219c113f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:36.346059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a83f28-ef6d-450b-bd2e-dddf219c113f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:36.357934 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 308/619] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:36.364264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a83f28-ef6d-450b-bd2e-dddf219c113f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:36.383256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:36.404582 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ac054-6f76-4b6f-bcf1-f4ee30bbd9b2 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:36.442758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09abf0-018e-4380-b78e-063a9908721e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:36.443453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09abf0-018e-4380-b78e-063a9908721e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:36.443740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09abf0-018e-4380-b78e-063a9908721e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:36.443997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09abf0-018e-4380-b78e-063a9908721e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:36.452056 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09abf0-018e-4380-b78e-063a9908721e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:36.453554 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09abf0-018e-4380-b78e-063a9908721e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:36.473874 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a83f28-ef6d-450b-bd2e-dddf219c113f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:36.486750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-2b09abf0-018e-4380-b78e-063a9908721e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:42:36.486750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:36.486750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:36.486750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:36.486750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:36.486750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:36.486750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:36.486750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:36.486750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:36.486750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:36.486750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:36.486750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:36.486750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:36.486750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:36.486750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:36.486750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:36.487689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:36.488680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:36.488680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:36.488680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:36.488680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:36.488680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:36.488680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:36.488680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:36.488680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:36.488680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:42:36.488680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:36.488680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 312/620] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 7 07:42:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:42:36.494172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09abf0-018e-4380-b78e-063a9908721e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:36.514424 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ac71c-888b-41dd-a87d-e3df4d7eebf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:36.515047 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ac71c-888b-41dd-a87d-e3df4d7eebf8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:36.515323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ac71c-888b-41dd-a87d-e3df4d7eebf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:36.515596 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ac71c-888b-41dd-a87d-e3df4d7eebf8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:36.524491 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ac71c-888b-41dd-a87d-e3df4d7eebf8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:36.527387 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ac71c-888b-41dd-a87d-e3df4d7eebf8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:36.542059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-836ac71c-888b-41dd-a87d-e3df4d7eebf8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 07:42:36.542059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:36.542059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:36.542059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:36.542059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:36.542059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:36.542059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:36.542059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:36.542059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:36.542059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:36.542059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:36.542059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:36.542059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:36.542059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:36.542059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:36.542059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:36.544401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:36.546760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:36.546760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:36.546760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:36.546760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:36.546760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:36.546760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:36.546760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:36.546760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:36.546760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 07:42:36.546760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:36.546760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 309/621] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:42:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:42:36.555269 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ac71c-888b-41dd-a87d-e3df4d7eebf8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:36.558542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:36.624450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:36.655222 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ac71c-888b-41dd-a87d-e3df4d7eebf8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:36.665448 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09abf0-018e-4380-b78e-063a9908721e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:36.711975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669295b-2c88-4e9a-ac5b-d1bc08e4a10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:36.713378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669295b-2c88-4e9a-ac5b-d1bc08e4a10a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:36.713921 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669295b-2c88-4e9a-ac5b-d1bc08e4a10a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:36.715342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669295b-2c88-4e9a-ac5b-d1bc08e4a10a None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:36.716214 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669295b-2c88-4e9a-ac5b-d1bc08e4a10a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:36.717370 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669295b-2c88-4e9a-ac5b-d1bc08e4a10a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:36.719679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669295b-2c88-4e9a-ac5b-d1bc08e4a10a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:36.735954 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612b921-d4f8-42cf-a309-255681042634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:36.736590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612b921-d4f8-42cf-a309-255681042634 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:36.736865 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612b921-d4f8-42cf-a309-255681042634 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:36.737136 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612b921-d4f8-42cf-a309-255681042634 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:36.737858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612b921-d4f8-42cf-a309-255681042634 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:36.738161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612b921-d4f8-42cf-a309-255681042634 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:36.739599 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612b921-d4f8-42cf-a309-255681042634 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:36.757751 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 310/622] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Sep 7 07:42:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:36.766193 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669295b-2c88-4e9a-ac5b-d1bc08e4a10a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:36.788195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 313/623] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 7 07:42:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:36.794930 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612b921-d4f8-42cf-a309-255681042634 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:36.930463 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669295b-2c88-4e9a-ac5b-d1bc08e4a10a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:36.945164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:36.949838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612b921-d4f8-42cf-a309-255681042634 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:36.962441 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:36.972599 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45e87e-63bc-4ec2-9c4d-48b042522fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:36.973563 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45e87e-63bc-4ec2-9c4d-48b042522fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:36.974045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45e87e-63bc-4ec2-9c4d-48b042522fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:36.974506 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45e87e-63bc-4ec2-9c4d-48b042522fd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:36.985715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45e87e-63bc-4ec2-9c4d-48b042522fd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:36.987559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45e87e-63bc-4ec2-9c4d-48b042522fd6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:37.001929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e37646-c0f6-4548-be8a-48e07b79f8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:37.002551 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e37646-c0f6-4548-be8a-48e07b79f8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:37.002833 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e37646-c0f6-4548-be8a-48e07b79f8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:37.003106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e37646-c0f6-4548-be8a-48e07b79f8f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:37.011182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-5f45e87e-63bc-4ec2-9c4d-48b042522fd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:37.011182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:37.011182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:37.011182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:37.011182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:37.011182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:37.011182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:37.011182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:37.011182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:37.011182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:37.011182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:37.011182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:37.011182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:37.011182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:37.011182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:37.011182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:37.012204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:37.013085 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:37.013085 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:37.013085 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:37.013085 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 311/624] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:37.014614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af630f5b-fc28-4dd8-afb0-d0d662cc5b28 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:37.014614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af630f5b-fc28-4dd8-afb0-d0d662cc5b28 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:37.014614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af630f5b-fc28-4dd8-afb0-d0d662cc5b28 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:37.015935 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 312/625] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:37.023486 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af630f5b-fc28-4dd8-afb0-d0d662cc5b28 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:37.088847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e37646-c0f6-4548-be8a-48e07b79f8f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:37.090339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e37646-c0f6-4548-be8a-48e07b79f8f3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:37.136112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af630f5b-fc28-4dd8-afb0-d0d662cc5b28 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:37.174976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cec8a-32d9-4e12-94b3-ad11ffb16516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:37.175945 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cec8a-32d9-4e12-94b3-ad11ffb16516 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:37.176601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cec8a-32d9-4e12-94b3-ad11ffb16516 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:37.177064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cec8a-32d9-4e12-94b3-ad11ffb16516 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:37.177962 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5cec8a-32d9-4e12-94b3-ad11ffb16516 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:37.178487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5cec8a-32d9-4e12-94b3-ad11ffb16516 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:37.180125 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5cec8a-32d9-4e12-94b3-ad11ffb16516 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:37.194724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 313/626] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:37.196398 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93decb62-0aa6-411c-8be5-10d6e0eb003f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:37.196906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93decb62-0aa6-411c-8be5-10d6e0eb003f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:37.197418 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93decb62-0aa6-411c-8be5-10d6e0eb003f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:37.201420 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:37.213389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 314/627] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:37.214513 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e37646-c0f6-4548-be8a-48e07b79f8f3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:37.219931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-93decb62-0aa6-411c-8be5-10d6e0eb003f None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:37.221481 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-93decb62-0aa6-411c-8be5-10d6e0eb003f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:37.264489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:37.294699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 314/628] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:37.296334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93decb62-0aa6-411c-8be5-10d6e0eb003f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:37.329655 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e37646-c0f6-4548-be8a-48e07b79f8f3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:37.367803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af14d7-d721-4f8c-8c0f-1f5790c911d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:37.368381 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af14d7-d721-4f8c-8c0f-1f5790c911d5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:37.368655 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af14d7-d721-4f8c-8c0f-1f5790c911d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:37.368927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af14d7-d721-4f8c-8c0f-1f5790c911d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:37.375660 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af14d7-d721-4f8c-8c0f-1f5790c911d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:37.377236 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af14d7-d721-4f8c-8c0f-1f5790c911d5 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:37.389344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-18af14d7-d721-4f8c-8c0f-1f5790c911d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:37.389344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:37.389344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:37.389344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:37.389344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:37.389344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:37.389344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:37.389344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:37.389344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:37.389344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:37.389344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:37.389344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:37.389344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:37.389344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:37.389344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:37.390208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:37.391067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:37.391067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:37.391067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:37.391067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:37.391067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:37.391067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:37.391067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:37.391067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:37.391067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 315/629] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:37.394095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d439b-a309-4159-9a9f-86e6250aa478 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:37.394203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d439b-a309-4159-9a9f-86e6250aa478 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:37.394461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d439b-a309-4159-9a9f-86e6250aa478 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:37.396556 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 316/630] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:37.397613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d490a-a29a-4376-b278-1690bc4ae20d None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:37.397896 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d490a-a29a-4376-b278-1690bc4ae20d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:37.398169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d490a-a29a-4376-b278-1690bc4ae20d None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:37.403622 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 317/631] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:37.403622 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d490a-a29a-4376-b278-1690bc4ae20d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:37.413841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93decb62-0aa6-411c-8be5-10d6e0eb003f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:37.453393 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ec843-3496-4270-be29-e2ffe6b45a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:37.454070 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ec843-3496-4270-be29-e2ffe6b45a9b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:37.454344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ec843-3496-4270-be29-e2ffe6b45a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:37.454802 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ec843-3496-4270-be29-e2ffe6b45a9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:37.465675 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ec843-3496-4270-be29-e2ffe6b45a9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:37.467100 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ec843-3496-4270-be29-e2ffe6b45a9b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:37.471694 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:37.481813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-970ec843-3496-4270-be29-e2ffe6b45a9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:37.481813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:37.481813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:37.481813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:37.481813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:37.481813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:37.481813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:37.481813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:37.481813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:37.481813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:37.481813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:37.481813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:37.481813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:37.481813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:37.481813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:37.481813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:37.482739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:37.483603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:37.483603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:37.483603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:37.483603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 315/632] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:37.485778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0b32f-d036-4644-8178-a951113aa535 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:37.485778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0b32f-d036-4644-8178-a951113aa535 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:37.485778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0b32f-d036-4644-8178-a951113aa535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:37.504846 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-17e0b32f-d036-4644-8178-a951113aa535 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:37.506026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-17e0b32f-d036-4644-8178-a951113aa535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:37.524164 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:37.527097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d490a-a29a-4376-b278-1690bc4ae20d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:37.562386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4224f4a-7329-4a52-aca1-d356af8c8d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:37.563035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4224f4a-7329-4a52-aca1-d356af8c8d05 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:37.563871 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4224f4a-7329-4a52-aca1-d356af8c8d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:37.564171 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4224f4a-7329-4a52-aca1-d356af8c8d05 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:37.564850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4224f4a-7329-4a52-aca1-d356af8c8d05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:37.565155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4224f4a-7329-4a52-aca1-d356af8c8d05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:37.566773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4224f4a-7329-4a52-aca1-d356af8c8d05 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:37.578692 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 316/633] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:37.581445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eb3fb-cc48-43af-8125-3010cde07bd6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:37.581729 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eb3fb-cc48-43af-8125-3010cde07bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:37.581995 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eb3fb-cc48-43af-8125-3010cde07bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:37.587734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 318/634] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:37.587734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4224f4a-7329-4a52-aca1-d356af8c8d05 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:37.605586 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-196eb3fb-cc48-43af-8125-3010cde07bd6 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:37.607001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-196eb3fb-cc48-43af-8125-3010cde07bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:37.680005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 317/635] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:37.680819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196eb3fb-cc48-43af-8125-3010cde07bd6 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:37.707035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4224f4a-7329-4a52-aca1-d356af8c8d05 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:37.731944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:37.756613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e0339-d1d4-445e-bec1-736e1e8bff8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:37.757762 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e0339-d1d4-445e-bec1-736e1e8bff8b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:37.758243 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e0339-d1d4-445e-bec1-736e1e8bff8b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:37.758946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e0339-d1d4-445e-bec1-736e1e8bff8b None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:37.760513 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e0339-d1d4-445e-bec1-736e1e8bff8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:37.760829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e0339-d1d4-445e-bec1-736e1e8bff8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:37.762227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e0339-d1d4-445e-bec1-736e1e8bff8b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:37.776864 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 319/636] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:37.777984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e0339-d1d4-445e-bec1-736e1e8bff8b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:37.780541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:37.808570 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196eb3fb-cc48-43af-8125-3010cde07bd6 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:37.855713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66e165-5c80-425a-a132-852939b30c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:37.856879 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66e165-5c80-425a-a132-852939b30c12 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:37.858123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66e165-5c80-425a-a132-852939b30c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:37.858621 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66e165-5c80-425a-a132-852939b30c12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:37.869856 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66e165-5c80-425a-a132-852939b30c12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:37.874864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-1c66e165-5c80-425a-a132-852939b30c12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:37.874864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:37.874864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:37.874864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:37.874864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:37.874864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:37.874864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:37.874864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:37.874864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:37.874864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:37.874864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:37.874864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:37.874864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:37.874864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:37.874864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:37.874864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:37.876354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:37.876354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:37.876354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:37.876354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:37.876354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:37.876354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:37.876354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:37.876354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:37.876354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:37.876354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:37.876354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:37.876354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:37.876354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:37.876354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:37.876354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:37.876354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:37.877186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:37.877186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:37.877186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:37.877186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:37.877186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:37.877186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:37.877186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:37.877186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:37.877186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:37.877186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:37.877186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:37.877186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:37.879387 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 318/637] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:37.879387 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66e165-5c80-425a-a132-852939b30c12 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:37.890875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e0339-d1d4-445e-bec1-736e1e8bff8b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:37.931488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d283fa-2aa1-4d6f-be36-fcaf25ef60aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:37.931930 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d283fa-2aa1-4d6f-be36-fcaf25ef60aa None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:37.932415 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d283fa-2aa1-4d6f-be36-fcaf25ef60aa None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:37.932658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d283fa-2aa1-4d6f-be36-fcaf25ef60aa None admin] PATH_INFO: `/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/3ff3bf8da732489b8b3a8fbe5aa22f21/roles/e7960e6d957f4a1fa6547565db026f24` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:37.948632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d283fa-2aa1-4d6f-be36-fcaf25ef60aa None admin] RBAC: Authorizing `identity:create_grant(project_id=9a949b959e584474aafb4a05ebbabcc4, user_id=3ff3bf8da732489b8b3a8fbe5aa22f21, role_id=e7960e6d957f4a1fa6547565db026f24)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:37.950078 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d283fa-2aa1-4d6f-be36-fcaf25ef60aa None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:37.964634 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 320/638] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:42:37 2022] PUT /identity/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/3ff3bf8da732489b8b3a8fbe5aa22f21/roles/e7960e6d957f4a1fa6547565db026f24 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:42:37.965442 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d283fa-2aa1-4d6f-be36-fcaf25ef60aa None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:37.991728 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66e165-5c80-425a-a132-852939b30c12 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:37.996578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:38.034905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32af52-e9a5-4d3d-bf24-1afae5102450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:38.035545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32af52-e9a5-4d3d-bf24-1afae5102450 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:38.035827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32af52-e9a5-4d3d-bf24-1afae5102450 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:38.036119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32af52-e9a5-4d3d-bf24-1afae5102450 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:38.043921 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32af52-e9a5-4d3d-bf24-1afae5102450 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:38.045420 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32af52-e9a5-4d3d-bf24-1afae5102450 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:38.057997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-9b32af52-e9a5-4d3d-bf24-1afae5102450 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:38.057997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:38.057997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:38.057997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:38.057997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:38.057997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:38.057997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:38.057997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:38.057997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:38.057997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:38.057997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:38.057997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:38.057997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:38.057997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:38.057997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:38.059091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:38.059977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:38.059977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:38.059977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:38.059977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:38.059977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:38.059977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:38.059977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:38.059977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:38.059977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 319/639] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:38.059977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32af52-e9a5-4d3d-bf24-1afae5102450 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:38.066782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:38.083193 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d283fa-2aa1-4d6f-be36-fcaf25ef60aa None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:38.121146 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ac3d8-9f52-47b1-ba17-3fab284a87b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:38.121802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ac3d8-9f52-47b1-ba17-3fab284a87b7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:38.122070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ac3d8-9f52-47b1-ba17-3fab284a87b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:38.122352 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ac3d8-9f52-47b1-ba17-3fab284a87b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:38.140120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ac3d8-9f52-47b1-ba17-3fab284a87b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:38.142905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-836ac3d8-9f52-47b1-ba17-3fab284a87b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:38.142905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:38.142905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:38.142905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:38.142905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:38.142905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:38.142905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:38.142905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:38.142905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:38.142905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:38.142905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:38.142905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:38.142905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:38.142905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:38.142905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:38.142905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:38.143911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:38.143911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:38.143911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:38.143911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:38.143911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:38.143911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:38.143911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:38.143911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:38.143911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:38.143911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:38.143911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:38.143911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:38.143911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:38.143911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:38.143911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:38.143911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:38.144774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:38.144774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:38.144774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:38.144774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:38.144774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:38.144774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:38.144774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:38.144774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:38.144774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:38.144774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:38.144774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:38.144774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:38.144774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 321/640] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:38.145452 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ac3d8-9f52-47b1-ba17-3fab284a87b7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:38.186270 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32af52-e9a5-4d3d-bf24-1afae5102450 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:38.221806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913043a7-4781-4ea3-bc44-88dacf160c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:38.222419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913043a7-4781-4ea3-bc44-88dacf160c6f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:38.222696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913043a7-4781-4ea3-bc44-88dacf160c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:38.222960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913043a7-4781-4ea3-bc44-88dacf160c6f None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:38.223911 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913043a7-4781-4ea3-bc44-88dacf160c6f None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:38.224250 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913043a7-4781-4ea3-bc44-88dacf160c6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:38.225603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913043a7-4781-4ea3-bc44-88dacf160c6f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:38.239229 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 320/641] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:38.239229 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913043a7-4781-4ea3-bc44-88dacf160c6f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:38.247127 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:38.261095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ac3d8-9f52-47b1-ba17-3fab284a87b7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:38.298998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8b58e-4ac9-4e8e-9f0a-d240944b0c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:38.299634 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8b58e-4ac9-4e8e-9f0a-d240944b0c33 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:38.299909 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8b58e-4ac9-4e8e-9f0a-d240944b0c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:38.300211 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8b58e-4ac9-4e8e-9f0a-d240944b0c33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:38.302471 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:38.311685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8b58e-4ac9-4e8e-9f0a-d240944b0c33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:38.316657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-3cf8b58e-4ac9-4e8e-9f0a-d240944b0c33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:38.316657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:38.316657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:38.316657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:38.316657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:38.316657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:38.316657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:38.316657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:38.316657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:38.316657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:38.316657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:38.316657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:38.316657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:38.316657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:38.316657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:38.316657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:38.317628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:38.317628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:38.317628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:38.317628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:38.317628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:38.317628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:38.317628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:38.317628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:38.317628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:38.317628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:38.317628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:38.317628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:38.317628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:38.317628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:38.317628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:38.317628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:38.321107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:38.321107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:38.321107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:38.321107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:38.321107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:38.321107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:38.321107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:38.321107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:38.321107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:38.321107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:38.321107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:38.321107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:38.321107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 322/642] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:38.321107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8b58e-4ac9-4e8e-9f0a-d240944b0c33 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:38.350322 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913043a7-4781-4ea3-bc44-88dacf160c6f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:38.388519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb41429-aadf-468f-a213-8f4fc9c2b758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:38.389156 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb41429-aadf-468f-a213-8f4fc9c2b758 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:38.389438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb41429-aadf-468f-a213-8f4fc9c2b758 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:38.389713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb41429-aadf-468f-a213-8f4fc9c2b758 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:38.390399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb41429-aadf-468f-a213-8f4fc9c2b758 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:38.390707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb41429-aadf-468f-a213-8f4fc9c2b758 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:38.392602 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb41429-aadf-468f-a213-8f4fc9c2b758 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:38.405168 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 321/643] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:38.406129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb41429-aadf-468f-a213-8f4fc9c2b758 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:38.437957 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8b58e-4ac9-4e8e-9f0a-d240944b0c33 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:38.473939 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47aac47-f3c0-4b64-a3b8-a313f2760464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:38.474576 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47aac47-f3c0-4b64-a3b8-a313f2760464 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:38.474857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47aac47-f3c0-4b64-a3b8-a313f2760464 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:38.475134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47aac47-f3c0-4b64-a3b8-a313f2760464 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:38.475791 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47aac47-f3c0-4b64-a3b8-a313f2760464 None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:38.476271 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47aac47-f3c0-4b64-a3b8-a313f2760464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:38.477584 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47aac47-f3c0-4b64-a3b8-a313f2760464 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:38.487359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 323/644] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:38.488201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47aac47-f3c0-4b64-a3b8-a313f2760464 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:38.503058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:38.523928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb41429-aadf-468f-a213-8f4fc9c2b758 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:38.559661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:38.572489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c74a33-a7f0-4914-9365-d63b77e78b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:38.573107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c74a33-a7f0-4914-9365-d63b77e78b03 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:38.573386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c74a33-a7f0-4914-9365-d63b77e78b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:38.573658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c74a33-a7f0-4914-9365-d63b77e78b03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:38.584149 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c74a33-a7f0-4914-9365-d63b77e78b03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:38.585582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c74a33-a7f0-4914-9365-d63b77e78b03 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:38.599299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47aac47-f3c0-4b64-a3b8-a313f2760464 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:38.601126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-e5c74a33-a7f0-4914-9365-d63b77e78b03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:38.601126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:38.601126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:38.601126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:38.601126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:38.601126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:38.601126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:38.601126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:38.601126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:38.601126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:38.601126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:38.601126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:38.601126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:38.601126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:38.601126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:38.601126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:38.602038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:38.603108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:38.603108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:38.603108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:38.603108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 322/645] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:38.603108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c74a33-a7f0-4914-9365-d63b77e78b03 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:38.638785 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509cedc-4aaf-4990-9038-dad96d342c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:38.639386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509cedc-4aaf-4990-9038-dad96d342c5f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:38.639661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509cedc-4aaf-4990-9038-dad96d342c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:38.639928 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509cedc-4aaf-4990-9038-dad96d342c5f None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:38.640599 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509cedc-4aaf-4990-9038-dad96d342c5f None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:38.640897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509cedc-4aaf-4990-9038-dad96d342c5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:38.643034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509cedc-4aaf-4990-9038-dad96d342c5f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:38.651467 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 324/646] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:38.652985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509cedc-4aaf-4990-9038-dad96d342c5f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:38.725809 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c74a33-a7f0-4914-9365-d63b77e78b03 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:38.739701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:38.767994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f8e37-501f-4205-99f6-91429dd2102b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:38.768726 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f8e37-501f-4205-99f6-91429dd2102b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:38.769004 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f8e37-501f-4205-99f6-91429dd2102b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:38.769488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f8e37-501f-4205-99f6-91429dd2102b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:38.769982 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f8e37-501f-4205-99f6-91429dd2102b None admin] RBAC: Adding query filter params (role.id=e7960e6d957f4a1fa6547565db026f24, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=81d5689e9a344bafb74ca423ffd2b42a) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:38.770322 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f8e37-501f-4205-99f6-91429dd2102b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:38.771702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f8e37-501f-4205-99f6-91429dd2102b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:38.772366 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509cedc-4aaf-4990-9038-dad96d342c5f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:38.785846 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 323/647] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:42:38 2022] GET /identity/v3/role_assignments?user.id=81d5689e9a344bafb74ca423ffd2b42a&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=e7960e6d957f4a1fa6547565db026f24 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:38.785846 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f8e37-501f-4205-99f6-91429dd2102b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:38.820338 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f08ed2-7fae-4997-8fc8-bca7e3edb6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:38.821140 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f08ed2-7fae-4997-8fc8-bca7e3edb6d8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:38.821483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f08ed2-7fae-4997-8fc8-bca7e3edb6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:38.821958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f08ed2-7fae-4997-8fc8-bca7e3edb6d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:38.835410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f08ed2-7fae-4997-8fc8-bca7e3edb6d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:38.837128 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f08ed2-7fae-4997-8fc8-bca7e3edb6d8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:38.841289 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:38.858115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-46f08ed2-7fae-4997-8fc8-bca7e3edb6d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:38.858115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:38.858115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:38.858115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:38.858115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:38.858115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:38.858115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:38.858115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:38.858115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:38.858115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:38.858115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:38.858115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:38.858115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:38.858115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:38.858115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:38.858115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:38.859344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:38.860717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:38.860717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:38.860717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:38.863249 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 325/648] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:38.863249 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f08ed2-7fae-4997-8fc8-bca7e3edb6d8 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:38.927165 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f8e37-501f-4205-99f6-91429dd2102b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:38.972773 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad2fb9-3e08-41ab-9a67-69e2aa52e838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:38.973443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad2fb9-3e08-41ab-9a67-69e2aa52e838 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:38.973824 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad2fb9-3e08-41ab-9a67-69e2aa52e838 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:38.974100 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad2fb9-3e08-41ab-9a67-69e2aa52e838 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:38.974743 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad2fb9-3e08-41ab-9a67-69e2aa52e838 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:38.975044 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad2fb9-3e08-41ab-9a67-69e2aa52e838 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:38.976583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad2fb9-3e08-41ab-9a67-69e2aa52e838 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:38.988525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f08ed2-7fae-4997-8fc8-bca7e3edb6d8 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:38.992036 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 324/649] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:38.992963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad2fb9-3e08-41ab-9a67-69e2aa52e838 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:39.018626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:39.036649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec20e47-bd26-42f5-8ff9-361743193388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:39.037627 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec20e47-bd26-42f5-8ff9-361743193388 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:39.037627 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec20e47-bd26-42f5-8ff9-361743193388 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:39.037810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec20e47-bd26-42f5-8ff9-361743193388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:39.051052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec20e47-bd26-42f5-8ff9-361743193388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:39.052838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec20e47-bd26-42f5-8ff9-361743193388 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:39.072239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-bec20e47-bd26-42f5-8ff9-361743193388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:39.072239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:39.072239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:39.072239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:39.072239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:39.072239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:39.072239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:39.072239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:39.072239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:39.072239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:39.072239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:39.072239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:39.072239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:39.072239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:39.072239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:39.072239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:39.073235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:39.074090 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:39.074090 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:39.074090 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:39.074090 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 326/650] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:39.074652 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec20e47-bd26-42f5-8ff9-361743193388 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:39.122675 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:39.129145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad2fb9-3e08-41ab-9a67-69e2aa52e838 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:39.171987 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cdae8-cf15-4a2a-b9c0-3d5b8ebbaa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:39.173651 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cdae8-cf15-4a2a-b9c0-3d5b8ebbaa6b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:39.174211 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cdae8-cf15-4a2a-b9c0-3d5b8ebbaa6b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:39.174709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cdae8-cf15-4a2a-b9c0-3d5b8ebbaa6b None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:39.175565 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cdae8-cf15-4a2a-b9c0-3d5b8ebbaa6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:39.176091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cdae8-cf15-4a2a-b9c0-3d5b8ebbaa6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:39.177677 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cdae8-cf15-4a2a-b9c0-3d5b8ebbaa6b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:39.190221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 325/651] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:39.191395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cdae8-cf15-4a2a-b9c0-3d5b8ebbaa6b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:39.201047 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec20e47-bd26-42f5-8ff9-361743193388 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:39.238600 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b20227-1b49-4877-af91-8c41cea88076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:39.239146 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b20227-1b49-4877-af91-8c41cea88076 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:39.239420 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b20227-1b49-4877-af91-8c41cea88076 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:39.239685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b20227-1b49-4877-af91-8c41cea88076 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:39.248002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b20227-1b49-4877-af91-8c41cea88076 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:39.249544 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b20227-1b49-4877-af91-8c41cea88076 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:39.262590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-e6b20227-1b49-4877-af91-8c41cea88076 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 07:42:39.262590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:39.262590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:39.262590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:39.262590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:39.262590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:39.262590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:39.262590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:39.262590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:39.262590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:39.262590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:39.262590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:39.262590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:39.262590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:39.262590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:39.262590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:39.263746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:39.264975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:39.264975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:39.264975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:39.264975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:39.264975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:39.264975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:39.264975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:39.264975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:39.264975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 07:42:39.264975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:39.264975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 327/652] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:39.264975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b20227-1b49-4877-af91-8c41cea88076 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:39.312434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:39.320998 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cdae8-cf15-4a2a-b9c0-3d5b8ebbaa6b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:39.357828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df2fa2-ea7d-455b-9d60-4dd44a76f6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:39.358715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df2fa2-ea7d-455b-9d60-4dd44a76f6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:39.359086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df2fa2-ea7d-455b-9d60-4dd44a76f6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:39.359453 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df2fa2-ea7d-455b-9d60-4dd44a76f6d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:39.360846 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df2fa2-ea7d-455b-9d60-4dd44a76f6d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:39.361435 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df2fa2-ea7d-455b-9d60-4dd44a76f6d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:39.362964 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df2fa2-ea7d-455b-9d60-4dd44a76f6d4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:39.375777 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 326/653] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:39.376927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df2fa2-ea7d-455b-9d60-4dd44a76f6d4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:39.393108 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b20227-1b49-4877-af91-8c41cea88076 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:39.406171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:39.432940 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934877f-1546-4502-bfd2-6e57f2c02547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:39.433578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934877f-1546-4502-bfd2-6e57f2c02547 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:39.433857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934877f-1546-4502-bfd2-6e57f2c02547 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:39.434130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934877f-1546-4502-bfd2-6e57f2c02547 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:39.443603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934877f-1546-4502-bfd2-6e57f2c02547 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:39.445466 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934877f-1546-4502-bfd2-6e57f2c02547 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:39.458744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-1934877f-1546-4502-bfd2-6e57f2c02547 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 07:42:39.458744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:39.458744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:39.458744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:39.458744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:39.458744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:39.458744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:39.458744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:39.458744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:39.458744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:39.458744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:39.458744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:39.458744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:39.458744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:39.458744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:39.458744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:39.459695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:39.460648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:39.460648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:39.460648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:39.460648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:39.460648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:39.460648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:39.460648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:39.460648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:39.460648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 07:42:39.460648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:39.464739 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 328/654] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:39.465561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934877f-1546-4502-bfd2-6e57f2c02547 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:39.502206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df2fa2-ea7d-455b-9d60-4dd44a76f6d4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:39.538745 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed6f0d-514c-44d6-b3d2-115eb2aaa96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:39.539791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed6f0d-514c-44d6-b3d2-115eb2aaa96b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:39.540315 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed6f0d-514c-44d6-b3d2-115eb2aaa96b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:39.540781 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed6f0d-514c-44d6-b3d2-115eb2aaa96b None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:39.541694 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed6f0d-514c-44d6-b3d2-115eb2aaa96b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:39.542205 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed6f0d-514c-44d6-b3d2-115eb2aaa96b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:39.544033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed6f0d-514c-44d6-b3d2-115eb2aaa96b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:39.565489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 327/655] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Sep 7 07:42:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:39.566628 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed6f0d-514c-44d6-b3d2-115eb2aaa96b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:39.575570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:39.584580 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934877f-1546-4502-bfd2-6e57f2c02547 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:39.627126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703580f-c482-4260-a41f-e33dac70bec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:39.628305 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703580f-c482-4260-a41f-e33dac70bec9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:39.628806 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703580f-c482-4260-a41f-e33dac70bec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:39.629679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703580f-c482-4260-a41f-e33dac70bec9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:39.640683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703580f-c482-4260-a41f-e33dac70bec9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:39.643287 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703580f-c482-4260-a41f-e33dac70bec9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:39.658913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-4703580f-c482-4260-a41f-e33dac70bec9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 07:42:39.658913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:39.658913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:39.658913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:39.658913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:39.658913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:39.658913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:39.658913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:39.658913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:39.658913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:39.658913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:39.658913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:39.658913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:39.658913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:39.658913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:39.658913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:39.659879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:39.660835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:39.660835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:39.660835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:39.660835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:39.660835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:39.660835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:39.660835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:39.660835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:39.660835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 07:42:39.660835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:39.663469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 329/656] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:39.664567 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703580f-c482-4260-a41f-e33dac70bec9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:39.671922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:39.701470 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed6f0d-514c-44d6-b3d2-115eb2aaa96b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:39.765158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7576767-5ea9-49f9-b54e-0a555b20cacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:39.765727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7576767-5ea9-49f9-b54e-0a555b20cacd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:39.765994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7576767-5ea9-49f9-b54e-0a555b20cacd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:39.766309 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7576767-5ea9-49f9-b54e-0a555b20cacd None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:39.767044 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7576767-5ea9-49f9-b54e-0a555b20cacd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:39.767338 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7576767-5ea9-49f9-b54e-0a555b20cacd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:39.768956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7576767-5ea9-49f9-b54e-0a555b20cacd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:39.794039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703580f-c482-4260-a41f-e33dac70bec9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:39.825140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 328/657] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Sep 7 07:42:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:39.828323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7576767-5ea9-49f9-b54e-0a555b20cacd None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:39.839686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41e689-faec-4208-aed6-628c651e10a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:39.842612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41e689-faec-4208-aed6-628c651e10a3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:39.842986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41e689-faec-4208-aed6-628c651e10a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:39.844259 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41e689-faec-4208-aed6-628c651e10a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:39.889576 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:39.956320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41e689-faec-4208-aed6-628c651e10a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:39.959285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41e689-faec-4208-aed6-628c651e10a3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:40.104237 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7576767-5ea9-49f9-b54e-0a555b20cacd None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:40.146861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 330/658] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:40.147732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41e689-faec-4208-aed6-628c651e10a3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:40.159993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99feb0-5930-4fcb-b1df-00afbfd7872b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:40.160988 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99feb0-5930-4fcb-b1df-00afbfd7872b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:40.161263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99feb0-5930-4fcb-b1df-00afbfd7872b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:40.161585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99feb0-5930-4fcb-b1df-00afbfd7872b None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:40.163083 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99feb0-5930-4fcb-b1df-00afbfd7872b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:40.163725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99feb0-5930-4fcb-b1df-00afbfd7872b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:40.166071 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99feb0-5930-4fcb-b1df-00afbfd7872b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:40.177394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:40.199989 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 329/659] 10.222.0.26 () {56 vars in 1139 bytes} [Wed Sep 7 07:42:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:40.200856 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99feb0-5930-4fcb-b1df-00afbfd7872b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:40.241974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:40.290284 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41e689-faec-4208-aed6-628c651e10a3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:40.313518 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99feb0-5930-4fcb-b1df-00afbfd7872b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:40.326102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cda4a-7c04-41e6-b7d0-d7eceba12008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:40.326699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cda4a-7c04-41e6-b7d0-d7eceba12008 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:40.326974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cda4a-7c04-41e6-b7d0-d7eceba12008 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:40.327241 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cda4a-7c04-41e6-b7d0-d7eceba12008 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:40.351297 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787641b-485f-4463-8d61-2192b566832c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:40.351942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787641b-485f-4463-8d61-2192b566832c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:40.352259 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787641b-485f-4463-8d61-2192b566832c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:40.352541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787641b-485f-4463-8d61-2192b566832c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:40.363138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787641b-485f-4463-8d61-2192b566832c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:40.364516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787641b-485f-4463-8d61-2192b566832c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:40.380184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-4787641b-485f-4463-8d61-2192b566832c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:40.380184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:40.380184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:40.380184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:40.380184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:40.380184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:40.380184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:40.380184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:40.380184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:40.380184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:40.380184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:40.380184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:40.380184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:40.380184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:40.380184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:40.380184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:40.381123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:40.381994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:40.381994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:40.381994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:40.381994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 330/660] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:40.381994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ff47e-d485-4d5e-8781-8ba477e71d92 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:40.382273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ff47e-d485-4d5e-8781-8ba477e71d92 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:40.382342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ff47e-d485-4d5e-8781-8ba477e71d92 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:40.383774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 331/661] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:40.384494 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ff47e-d485-4d5e-8781-8ba477e71d92 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:40.412279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cda4a-7c04-41e6-b7d0-d7eceba12008 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:40.413819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cda4a-7c04-41e6-b7d0-d7eceba12008 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:40.442883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:40.471861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:40.503774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ff47e-d485-4d5e-8781-8ba477e71d92 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:40.526021 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 331/662] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:40.526021 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b5a7a-5d88-4dd6-98a5-f786d9516b08 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:40.526021 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b5a7a-5d88-4dd6-98a5-f786d9516b08 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:40.526021 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b5a7a-5d88-4dd6-98a5-f786d9516b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:40.541436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a62b5a7a-5d88-4dd6-98a5-f786d9516b08 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:40.541927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aab434-2d8f-4212-a331-16c12b8eaca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:40.542537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aab434-2d8f-4212-a331-16c12b8eaca0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:40.542636 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a62b5a7a-5d88-4dd6-98a5-f786d9516b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:40.542813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aab434-2d8f-4212-a331-16c12b8eaca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:40.543083 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aab434-2d8f-4212-a331-16c12b8eaca0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:40.616178 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 332/663] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:40.616769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b5a7a-5d88-4dd6-98a5-f786d9516b08 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:40.629670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aab434-2d8f-4212-a331-16c12b8eaca0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:40.631932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aab434-2d8f-4212-a331-16c12b8eaca0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:40.703166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:40.710131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:40.726205 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b5a7a-5d88-4dd6-98a5-f786d9516b08 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:40.764727 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920bdc1-6329-4197-9f3b-fe28c382426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:40.765488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920bdc1-6329-4197-9f3b-fe28c382426c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:40.765829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920bdc1-6329-4197-9f3b-fe28c382426c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:40.766164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920bdc1-6329-4197-9f3b-fe28c382426c None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:40.766934 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920bdc1-6329-4197-9f3b-fe28c382426c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:40.767233 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920bdc1-6329-4197-9f3b-fe28c382426c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:40.768754 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920bdc1-6329-4197-9f3b-fe28c382426c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:40.781121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 333/664] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:40.782632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920bdc1-6329-4197-9f3b-fe28c382426c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:40.785835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 332/665] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:40.794714 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aab434-2d8f-4212-a331-16c12b8eaca0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:40.895978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aab434-2d8f-4212-a331-16c12b8eaca0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:40.916202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920bdc1-6329-4197-9f3b-fe28c382426c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:40.934262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0441e7f-785c-4a2d-8a7e-73692afd82e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:40.934898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0441e7f-785c-4a2d-8a7e-73692afd82e4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:40.935173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0441e7f-785c-4a2d-8a7e-73692afd82e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:40.935443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0441e7f-785c-4a2d-8a7e-73692afd82e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:40.946408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0441e7f-785c-4a2d-8a7e-73692afd82e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:40.947827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0441e7f-785c-4a2d-8a7e-73692afd82e4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:40.954898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776ba42-a25b-4f0d-88a1-0867d48447a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:40.955532 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776ba42-a25b-4f0d-88a1-0867d48447a8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:40.955803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776ba42-a25b-4f0d-88a1-0867d48447a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:40.955935 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:40.956171 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776ba42-a25b-4f0d-88a1-0867d48447a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:40.965707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f0441e7f-785c-4a2d-8a7e-73692afd82e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:40.965707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:40.965707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:40.965707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:40.965707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:40.965707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:40.965707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:40.965707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:40.965707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:40.965707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:40.965707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:40.965707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:40.965707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:40.965707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:40.965707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:40.965707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:40.966798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:40.967702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:40.967702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:40.967702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:40.967873 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776ba42-a25b-4f0d-88a1-0867d48447a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:40.968788 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776ba42-a25b-4f0d-88a1-0867d48447a8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:40.970798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 333/666] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:40.971428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0441e7f-785c-4a2d-8a7e-73692afd82e4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:40.980689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:40.989311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-0776ba42-a25b-4f0d-88a1-0867d48447a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:40.989311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:40.989311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:40.989311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:40.989311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:40.989311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:40.989311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:40.989311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:40.989311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:40.989311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:40.989311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:40.989311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:40.989311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:40.989311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:40.989311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:40.989311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:40.990282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:40.995743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:40.995743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:40.995743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:40.995743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 334/667] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:40.995743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776ba42-a25b-4f0d-88a1-0867d48447a8 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:41.086063 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0441e7f-785c-4a2d-8a7e-73692afd82e4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:41.103673 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776ba42-a25b-4f0d-88a1-0867d48447a8 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:41.126392 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f067fdf-3a65-4ae8-a77c-fcd26c55fc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:41.127117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f067fdf-3a65-4ae8-a77c-fcd26c55fc81 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:41.127396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f067fdf-3a65-4ae8-a77c-fcd26c55fc81 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:41.127667 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f067fdf-3a65-4ae8-a77c-fcd26c55fc81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:41.135015 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f067fdf-3a65-4ae8-a77c-fcd26c55fc81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:41.136497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f067fdf-3a65-4ae8-a77c-fcd26c55fc81 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:41.141943 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9bd2e-6846-4276-8e3a-2c67cdb65762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:41.142823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9bd2e-6846-4276-8e3a-2c67cdb65762 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:41.143295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9bd2e-6846-4276-8e3a-2c67cdb65762 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:41.143703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9bd2e-6846-4276-8e3a-2c67cdb65762 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:41.148690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-4f067fdf-3a65-4ae8-a77c-fcd26c55fc81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:41.148690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:41.148690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:41.148690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:41.148690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:41.148690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:41.148690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:41.148690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:41.148690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:41.148690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:41.148690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:41.148690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:41.148690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:41.148690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:41.148690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:41.149730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:41.150615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:41.150615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:41.150615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:41.150615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:41.150615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:41.150615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:41.150615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:41.150615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:41.150615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 334/668] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:41.150615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f067fdf-3a65-4ae8-a77c-fcd26c55fc81 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:41.160566 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9bd2e-6846-4276-8e3a-2c67cdb65762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:41.166632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-5cc9bd2e-6846-4276-8e3a-2c67cdb65762 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:41.166632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:41.166632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:41.166632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:41.166632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:41.166632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:41.166632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:41.166632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:41.166632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:41.166632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:41.166632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:41.166632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:41.166632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:41.166632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:41.166632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:41.166632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:41.167659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:41.167659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:41.167659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:41.167659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:41.167659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:41.167659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:41.167659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:41.167659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:41.167659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:41.167659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:41.167659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:41.167659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:41.167659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:41.167659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:41.167659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:41.167659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:41.168590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:41.168590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:41.168590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:41.168590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:41.168590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:41.168590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:41.168590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:41.168590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:41.168590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:41.168590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:41.168590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:41.168590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:41.169448 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 335/669] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:41.170029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9bd2e-6846-4276-8e3a-2c67cdb65762 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:41.216172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:41.236375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:41.259140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f067fdf-3a65-4ae8-a77c-fcd26c55fc81 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:41.278074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9bd2e-6846-4276-8e3a-2c67cdb65762 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:41.292302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d9c4d-3af1-4602-9404-ffd7ebc79c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:41.292917 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d9c4d-3af1-4602-9404-ffd7ebc79c30 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:41.293197 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d9c4d-3af1-4602-9404-ffd7ebc79c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:41.293469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d9c4d-3af1-4602-9404-ffd7ebc79c30 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:41.294114 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d9c4d-3af1-4602-9404-ffd7ebc79c30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:41.294414 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d9c4d-3af1-4602-9404-ffd7ebc79c30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:41.295795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d9c4d-3af1-4602-9404-ffd7ebc79c30 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:41.307993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 335/670] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:41.308853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d9c4d-3af1-4602-9404-ffd7ebc79c30 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:41.320998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3aa07-38c8-4e18-9f45-b3f4c8bffd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:41.321599 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3aa07-38c8-4e18-9f45-b3f4c8bffd1c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:41.321873 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3aa07-38c8-4e18-9f45-b3f4c8bffd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:41.322138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3aa07-38c8-4e18-9f45-b3f4c8bffd1c None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:41.322771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb3aa07-38c8-4e18-9f45-b3f4c8bffd1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:41.323055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb3aa07-38c8-4e18-9f45-b3f4c8bffd1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:41.324448 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb3aa07-38c8-4e18-9f45-b3f4c8bffd1c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:41.336871 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 336/671] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:41.337506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3aa07-38c8-4e18-9f45-b3f4c8bffd1c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:41.412832 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d9c4d-3af1-4602-9404-ffd7ebc79c30 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:41.432039 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:41.439979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3aa07-38c8-4e18-9f45-b3f4c8bffd1c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:41.450121 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38613783-ae91-4009-820d-07fad3aee874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:41.450725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38613783-ae91-4009-820d-07fad3aee874 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:41.451004 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38613783-ae91-4009-820d-07fad3aee874 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:41.451276 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38613783-ae91-4009-820d-07fad3aee874 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:41.451966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38613783-ae91-4009-820d-07fad3aee874 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:41.452314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38613783-ae91-4009-820d-07fad3aee874 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:41.454372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38613783-ae91-4009-820d-07fad3aee874 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:41.458014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:41.465734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 336/672] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:41.466909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e1b04-a234-4bb3-9ee8-70f4c637fdac None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:41.467188 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e1b04-a234-4bb3-9ee8-70f4c637fdac None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:41.467453 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e1b04-a234-4bb3-9ee8-70f4c637fdac None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:41.468816 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 337/673] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:41.469339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e1b04-a234-4bb3-9ee8-70f4c637fdac None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:41.481363 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5886f-9ff5-4f67-bf66-58c8014308a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:41.481974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5886f-9ff5-4f67-bf66-58c8014308a5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:41.482251 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5886f-9ff5-4f67-bf66-58c8014308a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:41.482520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5886f-9ff5-4f67-bf66-58c8014308a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:41.483169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5886f-9ff5-4f67-bf66-58c8014308a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:41.483468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5886f-9ff5-4f67-bf66-58c8014308a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:41.484861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5886f-9ff5-4f67-bf66-58c8014308a5 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:41.493059 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 337/674] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:41.493831 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5886f-9ff5-4f67-bf66-58c8014308a5 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:41.570737 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e1b04-a234-4bb3-9ee8-70f4c637fdac None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:41.597123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5886f-9ff5-4f67-bf66-58c8014308a5 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:41.604156 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce306b-d6fc-4904-ade2-e15819b29661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:41.604771 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce306b-d6fc-4904-ade2-e15819b29661 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:41.605045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce306b-d6fc-4904-ade2-e15819b29661 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:41.605313 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce306b-d6fc-4904-ade2-e15819b29661 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:41.612658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce306b-d6fc-4904-ade2-e15819b29661 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:41.614073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce306b-d6fc-4904-ade2-e15819b29661 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:41.629290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-5bce306b-d6fc-4904-ade2-e15819b29661 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:41.629290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:41.629290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:41.629290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:41.629290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:41.629290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:41.629290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:41.629290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:41.629290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:41.629290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:41.629290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:41.629290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:41.629290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:41.629290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:41.629290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:41.630239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:41.631805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:41.631805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:41.631805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:41.631805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:41.631805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:41.631805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:41.631805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:41.631805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:41.631805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 338/675] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:41.631805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce306b-d6fc-4904-ade2-e15819b29661 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:41.632394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc1d7e-b8a9-4844-a4bc-86e7425ac3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:41.632394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc1d7e-b8a9-4844-a4bc-86e7425ac3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:41.632394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc1d7e-b8a9-4844-a4bc-86e7425ac3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:41.632637 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc1d7e-b8a9-4844-a4bc-86e7425ac3c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:41.640455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc1d7e-b8a9-4844-a4bc-86e7425ac3c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:41.642544 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc1d7e-b8a9-4844-a4bc-86e7425ac3c6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:41.653803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-86fc1d7e-b8a9-4844-a4bc-86e7425ac3c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:41.653803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:41.653803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:41.653803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:41.653803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:41.653803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:41.653803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:41.653803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:41.653803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:41.653803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:41.653803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:41.653803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:41.653803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:41.653803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:41.653803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:41.654731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:41.655623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:41.655623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:41.655623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:41.655623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:41.655623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:41.655623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:41.655623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:41.655623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:41.655623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 338/676] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:41.656948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131f5bd-f699-4c99-a3af-e18d86234c10 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:41.656948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131f5bd-f699-4c99-a3af-e18d86234c10 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:41.656948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131f5bd-f699-4c99-a3af-e18d86234c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:41.674437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:41.674826 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b131f5bd-f699-4c99-a3af-e18d86234c10 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:41.676006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b131f5bd-f699-4c99-a3af-e18d86234c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:41.710571 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:41.736571 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce306b-d6fc-4904-ade2-e15819b29661 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:41.743700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 339/677] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:41.744541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131f5bd-f699-4c99-a3af-e18d86234c10 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:41.781885 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8bd9d-dc0a-499e-b029-38ff01cb7ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:41.782590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8bd9d-dc0a-499e-b029-38ff01cb7ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:41.782863 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8bd9d-dc0a-499e-b029-38ff01cb7ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:41.783133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8bd9d-dc0a-499e-b029-38ff01cb7ae7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:41.783842 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8bd9d-dc0a-499e-b029-38ff01cb7ae7 None admin] RBAC: Adding query filter params (role.id=e7960e6d957f4a1fa6547565db026f24, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=9b0652c89c9945e2866512c9da4793ba) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:41.784172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8bd9d-dc0a-499e-b029-38ff01cb7ae7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:41.785573 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8bd9d-dc0a-499e-b029-38ff01cb7ae7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:41.791376 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 339/678] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:42:41 2022] GET /identity/v3/role_assignments?user.id=9b0652c89c9945e2866512c9da4793ba&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=e7960e6d957f4a1fa6547565db026f24 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:41.792179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8bd9d-dc0a-499e-b029-38ff01cb7ae7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:41.893759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131f5bd-f699-4c99-a3af-e18d86234c10 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:41.917181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8bd9d-dc0a-499e-b029-38ff01cb7ae7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:41.918975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:41.954790 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00c860-d8e3-4436-9cf3-6508d3a20d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:41.956030 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00c860-d8e3-4436-9cf3-6508d3a20d9c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:41.956575 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00c860-d8e3-4436-9cf3-6508d3a20d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:41.957426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00c860-d8e3-4436-9cf3-6508d3a20d9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:41.961800 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e399f1-0c96-4a64-8f09-0872d53384dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:41.962405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e399f1-0c96-4a64-8f09-0872d53384dc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:41.962679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e399f1-0c96-4a64-8f09-0872d53384dc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:41.962944 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e399f1-0c96-4a64-8f09-0872d53384dc None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:41.963635 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e399f1-0c96-4a64-8f09-0872d53384dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:41.963935 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e399f1-0c96-4a64-8f09-0872d53384dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:41.965375 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e399f1-0c96-4a64-8f09-0872d53384dc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:41.972457 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00c860-d8e3-4436-9cf3-6508d3a20d9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:41.973946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00c860-d8e3-4436-9cf3-6508d3a20d9c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:41.981266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 340/679] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:41.981946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e399f1-0c96-4a64-8f09-0872d53384dc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:41.996044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-0c00c860-d8e3-4436-9cf3-6508d3a20d9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:41.996044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:41.996044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:41.996044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:41.996044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:41.996044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:41.996044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:41.996044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:41.996044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:41.996044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:41.996044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:41.996044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:41.996044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:41.996044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:41.996044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:41.996044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:41.997069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:41.997931 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:41.997931 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:41.997931 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:41.999981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 340/680] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:42.001100 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00c860-d8e3-4436-9cf3-6508d3a20d9c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:42.035338 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:42.100877 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e399f1-0c96-4a64-8f09-0872d53384dc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:42.127476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00c860-d8e3-4436-9cf3-6508d3a20d9c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:42.134588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f83cee-abe5-4361-97cc-06b9248b5c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:42.135607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f83cee-abe5-4361-97cc-06b9248b5c6f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:42.135890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f83cee-abe5-4361-97cc-06b9248b5c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:42.136196 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f83cee-abe5-4361-97cc-06b9248b5c6f None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:42.136888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f83cee-abe5-4361-97cc-06b9248b5c6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:42.137550 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f83cee-abe5-4361-97cc-06b9248b5c6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:42.139328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f83cee-abe5-4361-97cc-06b9248b5c6f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:42.156192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 341/681] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:42.156192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f83cee-abe5-4361-97cc-06b9248b5c6f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:42.169419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a079098-7b6c-486b-a2e2-948a796f20d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:42.170155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a079098-7b6c-486b-a2e2-948a796f20d6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:42.170431 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a079098-7b6c-486b-a2e2-948a796f20d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:42.170701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a079098-7b6c-486b-a2e2-948a796f20d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:42.171412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a079098-7b6c-486b-a2e2-948a796f20d6 None admin] RBAC: Adding query filter params (role.id=e7960e6d957f4a1fa6547565db026f24, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=330a38822d33439191773e9a8b4cbe04) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:42.171715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a079098-7b6c-486b-a2e2-948a796f20d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:42.173668 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a079098-7b6c-486b-a2e2-948a796f20d6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:42.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:42.193718 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 341/682] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:42:42 2022] GET /identity/v3/role_assignments?user.id=330a38822d33439191773e9a8b4cbe04&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=e7960e6d957f4a1fa6547565db026f24 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:42.195295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a079098-7b6c-486b-a2e2-948a796f20d6 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:42.272540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f83cee-abe5-4361-97cc-06b9248b5c6f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:42.316683 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceae91f-0bf2-4d5c-b146-2e6086906dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:42.317421 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceae91f-0bf2-4d5c-b146-2e6086906dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:42.318269 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceae91f-0bf2-4d5c-b146-2e6086906dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:42.318269 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceae91f-0bf2-4d5c-b146-2e6086906dc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:42.318748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceae91f-0bf2-4d5c-b146-2e6086906dc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:42.319117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceae91f-0bf2-4d5c-b146-2e6086906dc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:42.320858 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceae91f-0bf2-4d5c-b146-2e6086906dc1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:42.334289 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 342/683] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:42.334787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:42.335638 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceae91f-0bf2-4d5c-b146-2e6086906dc1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:42.360853 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a079098-7b6c-486b-a2e2-948a796f20d6 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:42.402995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da24b39-f880-4c5b-813a-42148609c48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:42.403926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da24b39-f880-4c5b-813a-42148609c48f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:42.405141 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da24b39-f880-4c5b-813a-42148609c48f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:42.405636 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da24b39-f880-4c5b-813a-42148609c48f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:42.422074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da24b39-f880-4c5b-813a-42148609c48f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:42.428113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-8da24b39-f880-4c5b-813a-42148609c48f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:42:42.428113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:42.428113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:42.428113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:42.428113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:42.428113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:42.428113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:42.428113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:42.428113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:42.428113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:42.428113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:42.428113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:42.428113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:42.428113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:42.428113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:42.428113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:42.429074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:42.429074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:42.429074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:42.429074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:42.429074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:42.429074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:42.429074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:42.429074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:42.429074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:42.429074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:42.429074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:42.429074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:42.429074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:42.429074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:42.429074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:42.429074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:42.429925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:42.429925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:42.429925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:42.429925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:42.429925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:42.429925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:42.429925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:42.429925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:42.429925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:42.429925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:42.429925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:42:42.429925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:42.429925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 342/684] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:42:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:42.430623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da24b39-f880-4c5b-813a-42148609c48f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:42.443154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:42.468551 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceae91f-0bf2-4d5c-b146-2e6086906dc1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:42.508025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006e0fa-9370-4c25-b8c1-6879fbc9e0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:42.508723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006e0fa-9370-4c25-b8c1-6879fbc9e0fc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:42.509402 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006e0fa-9370-4c25-b8c1-6879fbc9e0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:42.509402 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006e0fa-9370-4c25-b8c1-6879fbc9e0fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:42.510027 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006e0fa-9370-4c25-b8c1-6879fbc9e0fc None admin] RBAC: Adding query filter params (role.id=e7960e6d957f4a1fa6547565db026f24, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=92699b512c064b05948afe597650fdb6) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:42.510340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006e0fa-9370-4c25-b8c1-6879fbc9e0fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:42.511768 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006e0fa-9370-4c25-b8c1-6879fbc9e0fc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:42.524339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 343/685] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:42:42 2022] GET /identity/v3/role_assignments?user.id=92699b512c064b05948afe597650fdb6&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=e7960e6d957f4a1fa6547565db026f24 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:42.525396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006e0fa-9370-4c25-b8c1-6879fbc9e0fc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:42.571648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da24b39-f880-4c5b-813a-42148609c48f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:42.620322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae140d-f5fb-464f-8635-befeeb81995d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:42.621316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae140d-f5fb-464f-8635-befeeb81995d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:42.621817 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae140d-f5fb-464f-8635-befeeb81995d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:42.622285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae140d-f5fb-464f-8635-befeeb81995d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:42.633767 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae140d-f5fb-464f-8635-befeeb81995d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:42.635911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae140d-f5fb-464f-8635-befeeb81995d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:42.651112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-1fae140d-f5fb-464f-8635-befeeb81995d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 07:42:42.651112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:42.651112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:42.651112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:42.651112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:42.651112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:42.651112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:42.651112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:42.651112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:42.651112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:42.651112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:42.651112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:42.651112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:42.651112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:42.651112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:42.651112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:42.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:42.653122 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:42.653122 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:42.653122 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:42.653122 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:42.653122 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:42.653122 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:42.653122 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:42.653122 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:42.653122 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 07:42:42.653122 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:42.656444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 343/686] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:42:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:42.663198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae140d-f5fb-464f-8635-befeeb81995d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:42.684609 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:42.717767 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006e0fa-9370-4c25-b8c1-6879fbc9e0fc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:42.758397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1e80c-d07f-410e-8b38-c78b42ea1f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:42.759013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1e80c-d07f-410e-8b38-c78b42ea1f55 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:42.759340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1e80c-d07f-410e-8b38-c78b42ea1f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:42.759619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1e80c-d07f-410e-8b38-c78b42ea1f55 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:42.760320 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1e80c-d07f-410e-8b38-c78b42ea1f55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:42.760625 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1e80c-d07f-410e-8b38-c78b42ea1f55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:42.762016 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1e80c-d07f-410e-8b38-c78b42ea1f55 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:42.776512 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 344/687] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:42.778337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1e80c-d07f-410e-8b38-c78b42ea1f55 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:42.799071 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:42.859825 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae140d-f5fb-464f-8635-befeeb81995d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:42.895559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1e80c-d07f-410e-8b38-c78b42ea1f55 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:42.898036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46efe89-80c0-4022-93c8-a357e4ea53ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:42.898895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46efe89-80c0-4022-93c8-a357e4ea53ca None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:42.899372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46efe89-80c0-4022-93c8-a357e4ea53ca None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:42.899841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46efe89-80c0-4022-93c8-a357e4ea53ca None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:42.900804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46efe89-80c0-4022-93c8-a357e4ea53ca None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:42.901317 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46efe89-80c0-4022-93c8-a357e4ea53ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:42.903067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46efe89-80c0-4022-93c8-a357e4ea53ca None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:42.924669 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 344/688] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Sep 7 07:42:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:42.933517 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46efe89-80c0-4022-93c8-a357e4ea53ca None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:42.937534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06804fcd-60a8-430b-876d-e6d0afdeb6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:42.938821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06804fcd-60a8-430b-876d-e6d0afdeb6ef None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:42.939302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06804fcd-60a8-430b-876d-e6d0afdeb6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:42.939787 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06804fcd-60a8-430b-876d-e6d0afdeb6ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:42.951647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06804fcd-60a8-430b-876d-e6d0afdeb6ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:42.953451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06804fcd-60a8-430b-876d-e6d0afdeb6ef None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:42.968865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-06804fcd-60a8-430b-876d-e6d0afdeb6ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:42.968865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:42.968865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:42.968865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:42.968865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:42.968865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:42.968865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:42.968865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:42.968865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:42.968865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:42.968865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:42.968865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:42.968865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:42.968865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:42.968865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:42.968865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:42.970568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:42.971722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:42.971722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:42.971722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:42.975521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 345/689] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:42:42.976775 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06804fcd-60a8-430b-876d-e6d0afdeb6ef None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:43.010041 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:43.041276 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:43.056449 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46efe89-80c0-4022-93c8-a357e4ea53ca None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:43.086413 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06804fcd-60a8-430b-876d-e6d0afdeb6ef None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:43.097757 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466e8b5-16e2-4ef6-8b78-724b7a6f347e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:43.098744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466e8b5-16e2-4ef6-8b78-724b7a6f347e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:43.099241 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466e8b5-16e2-4ef6-8b78-724b7a6f347e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:43.099759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466e8b5-16e2-4ef6-8b78-724b7a6f347e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:43.129730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6717f5-cbe2-4e14-b677-38c0d709a392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:43.130334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6717f5-cbe2-4e14-b677-38c0d709a392 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:43.130620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6717f5-cbe2-4e14-b677-38c0d709a392 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:43.130895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6717f5-cbe2-4e14-b677-38c0d709a392 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:43.131540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6717f5-cbe2-4e14-b677-38c0d709a392 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:43.131843 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6717f5-cbe2-4e14-b677-38c0d709a392 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:43.133301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6717f5-cbe2-4e14-b677-38c0d709a392 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:43.151616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 346/690] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:43.152897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6717f5-cbe2-4e14-b677-38c0d709a392 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:43.198342 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466e8b5-16e2-4ef6-8b78-724b7a6f347e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:43.200357 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466e8b5-16e2-4ef6-8b78-724b7a6f347e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:43.271579 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6717f5-cbe2-4e14-b677-38c0d709a392 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:43.306009 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:43.321517 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:43.337953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a5da7-aab9-4120-8c77-cb40030b2bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:43.339506 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a5da7-aab9-4120-8c77-cb40030b2bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:43.341157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a5da7-aab9-4120-8c77-cb40030b2bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:43.341668 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a5da7-aab9-4120-8c77-cb40030b2bf7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:43.356806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a5da7-aab9-4120-8c77-cb40030b2bf7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:43.358751 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a5da7-aab9-4120-8c77-cb40030b2bf7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:43.385685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-197a5da7-aab9-4120-8c77-cb40030b2bf7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:42:43.385685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:43.385685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:43.385685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:43.385685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:43.385685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:43.385685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:43.385685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:43.385685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:43.385685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:43.385685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:43.385685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:43.385685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:43.385685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:43.385685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:43.385685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:43.386684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:43.387608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:43.387608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:43.387608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:43.387608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:43.387608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:43.387608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:43.387608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:43.387608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:43.387608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:42:43.387608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:43.387608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 347/691] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 7 07:42:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:43.392884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 345/692] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:43.393461 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a5da7-aab9-4120-8c77-cb40030b2bf7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:43.401642 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466e8b5-16e2-4ef6-8b78-724b7a6f347e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:43.550322 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a5da7-aab9-4120-8c77-cb40030b2bf7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:43.581932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466e8b5-16e2-4ef6-8b78-724b7a6f347e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:43.620336 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb8d5e-a5bd-4007-9ebe-cf2d1c305700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:43.621303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb8d5e-a5bd-4007-9ebe-cf2d1c305700 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:43.621787 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb8d5e-a5bd-4007-9ebe-cf2d1c305700 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:43.622282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb8d5e-a5bd-4007-9ebe-cf2d1c305700 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:43.623232 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb8d5e-a5bd-4007-9ebe-cf2d1c305700 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:43.623742 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb8d5e-a5bd-4007-9ebe-cf2d1c305700 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:43.625461 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb8d5e-a5bd-4007-9ebe-cf2d1c305700 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:43.654352 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 348/693] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 7 07:42:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:43.658267 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c11509-bcd2-406d-81b9-fe699adf821f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:43.659169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c11509-bcd2-406d-81b9-fe699adf821f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:43.659719 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c11509-bcd2-406d-81b9-fe699adf821f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:43.660836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb8d5e-a5bd-4007-9ebe-cf2d1c305700 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:43.661714 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c11509-bcd2-406d-81b9-fe699adf821f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:43.684174 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c11509-bcd2-406d-81b9-fe699adf821f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:43.687424 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c11509-bcd2-406d-81b9-fe699adf821f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:43.714570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-79c11509-bcd2-406d-81b9-fe699adf821f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:43.714570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:43.714570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:43.714570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:43.714570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:43.714570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:43.714570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:43.714570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:43.714570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:43.714570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:43.714570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:43.714570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:43.714570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:43.714570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:43.714570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:43.714570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:43.715547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:43.716491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:43.716491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:43.716491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:43.723830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 346/694] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:43.732441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c11509-bcd2-406d-81b9-fe699adf821f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:43.742024 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:43.767743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:43.857399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb8d5e-a5bd-4007-9ebe-cf2d1c305700 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:43.880339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c11509-bcd2-406d-81b9-fe699adf821f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:43.915884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106ee73-652f-46af-92b7-076fe2edf0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:43.916663 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106ee73-652f-46af-92b7-076fe2edf0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:43.917050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106ee73-652f-46af-92b7-076fe2edf0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:43.917432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106ee73-652f-46af-92b7-076fe2edf0e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:43.918173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106ee73-652f-46af-92b7-076fe2edf0e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:43.918613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106ee73-652f-46af-92b7-076fe2edf0e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:43.920163 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106ee73-652f-46af-92b7-076fe2edf0e0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:43.926740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca48eaa-9a61-4627-8419-8b1b352aede4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:43.927343 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca48eaa-9a61-4627-8419-8b1b352aede4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:43.927623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca48eaa-9a61-4627-8419-8b1b352aede4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:43.927896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca48eaa-9a61-4627-8419-8b1b352aede4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:43.949056 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 347/695] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:43.955403 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106ee73-652f-46af-92b7-076fe2edf0e0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:44.039793 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:44.082298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106ee73-652f-46af-92b7-076fe2edf0e0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:44.123736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca48eaa-9a61-4627-8419-8b1b352aede4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:44.125682 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca48eaa-9a61-4627-8419-8b1b352aede4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:44.157134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995d103-9598-4023-a270-197af6185d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:44.159138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995d103-9598-4023-a270-197af6185d74 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:44.160388 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995d103-9598-4023-a270-197af6185d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:44.161034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995d103-9598-4023-a270-197af6185d74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:44.180783 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995d103-9598-4023-a270-197af6185d74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:44.182260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995d103-9598-4023-a270-197af6185d74 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:44.200861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-e995d103-9598-4023-a270-197af6185d74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:44.200861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:44.200861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:44.200861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:44.200861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:44.200861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:44.200861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:44.200861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:44.200861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:44.200861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:44.200861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:44.200861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:44.200861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:44.200861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:44.200861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:44.201840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:44.202753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:44.202753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:44.202753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:44.202753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:44.202753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:44.202753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:44.202753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:44.202753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:44.215280 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 348/696] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:42:44.222966 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995d103-9598-4023-a270-197af6185d74 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:44.234797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:44.295843 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 349/697] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:44.297171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649a123-48a7-428d-982d-48095cd66de1 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:44.297512 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649a123-48a7-428d-982d-48095cd66de1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:44.297792 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649a123-48a7-428d-982d-48095cd66de1 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:44.299314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 350/698] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:44.305278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649a123-48a7-428d-982d-48095cd66de1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:44.342812 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995d103-9598-4023-a270-197af6185d74 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:44.381440 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe93c3-3567-4d3a-a620-bddce8c68c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:44.382163 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe93c3-3567-4d3a-a620-bddce8c68c30 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:44.382439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe93c3-3567-4d3a-a620-bddce8c68c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:44.382764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe93c3-3567-4d3a-a620-bddce8c68c30 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:44.383437 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe93c3-3567-4d3a-a620-bddce8c68c30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:44.383790 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe93c3-3567-4d3a-a620-bddce8c68c30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:44.385332 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe93c3-3567-4d3a-a620-bddce8c68c30 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:44.388384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:44.400134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 349/699] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:44.401308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2f61b-032b-4a9f-a331-da04f019f263 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:44.401589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2f61b-032b-4a9f-a331-da04f019f263 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:44.401861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2f61b-032b-4a9f-a331-da04f019f263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:44.422429 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-95a2f61b-032b-4a9f-a331-da04f019f263 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:44.423661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-95a2f61b-032b-4a9f-a331-da04f019f263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:44.428794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649a123-48a7-428d-982d-48095cd66de1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:44.467695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9c595-6682-4da3-9025-60278829e36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:44.468790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9c595-6682-4da3-9025-60278829e36d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:44.469801 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9c595-6682-4da3-9025-60278829e36d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:44.470512 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9c595-6682-4da3-9025-60278829e36d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:44.478044 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:44.482634 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9c595-6682-4da3-9025-60278829e36d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:44.484615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9c595-6682-4da3-9025-60278829e36d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:44.503614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-8de9c595-6682-4da3-9025-60278829e36d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:44.503614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:44.503614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:44.503614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:44.503614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:44.503614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:44.503614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:44.503614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:44.503614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:44.503614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:44.503614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:44.503614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:44.503614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:44.503614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:44.503614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:44.503614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:44.504627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:44.505485 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:44.505485 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:44.505485 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:44.506002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 350/701] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:44.507032 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2f61b-032b-4a9f-a331-da04f019f263 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:44.509457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 351/701] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:44.511448 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9c595-6682-4da3-9025-60278829e36d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:44.672064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9c595-6682-4da3-9025-60278829e36d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:44.713340 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2f61b-032b-4a9f-a331-da04f019f263 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:44.747970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e13bba-236e-4dda-a817-d8035adf59bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:44.748619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e13bba-236e-4dda-a817-d8035adf59bd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:44.748897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e13bba-236e-4dda-a817-d8035adf59bd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:44.749171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e13bba-236e-4dda-a817-d8035adf59bd None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:44.749824 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e13bba-236e-4dda-a817-d8035adf59bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:44.750130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e13bba-236e-4dda-a817-d8035adf59bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:44.752109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e13bba-236e-4dda-a817-d8035adf59bd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:44.774000 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:44.782577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 352/702] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:44.787636 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e52f6-c98b-4b0f-ab47-5199ec9cb1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:44.788281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e52f6-c98b-4b0f-ab47-5199ec9cb1ef None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:44.788561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e52f6-c98b-4b0f-ab47-5199ec9cb1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:44.788833 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e52f6-c98b-4b0f-ab47-5199ec9cb1ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:44.789556 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e52f6-c98b-4b0f-ab47-5199ec9cb1ef None admin] RBAC: Adding query filter params (role.id=e7960e6d957f4a1fa6547565db026f24, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=3ff3bf8da732489b8b3a8fbe5aa22f21) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:44.789860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e52f6-c98b-4b0f-ab47-5199ec9cb1ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:44.791293 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e52f6-c98b-4b0f-ab47-5199ec9cb1ef None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:44.798758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 351/703] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:42:44 2022] GET /identity/v3/role_assignments?user.id=3ff3bf8da732489b8b3a8fbe5aa22f21&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=e7960e6d957f4a1fa6547565db026f24 => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:44.801158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e13bba-236e-4dda-a817-d8035adf59bd None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:44.853814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e52f6-c98b-4b0f-ab47-5199ec9cb1ef None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:44.868515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:44.923633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e13bba-236e-4dda-a817-d8035adf59bd None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:44.963859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe11fb-af5a-43a6-ac9e-af514b8faa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:44.964595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe11fb-af5a-43a6-ac9e-af514b8faa7e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:44.964883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe11fb-af5a-43a6-ac9e-af514b8faa7e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:44.965206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe11fb-af5a-43a6-ac9e-af514b8faa7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:44.973154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe11fb-af5a-43a6-ac9e-af514b8faa7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:44.974607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe11fb-af5a-43a6-ac9e-af514b8faa7e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:44.988794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-b5fe11fb-af5a-43a6-ac9e-af514b8faa7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:44.988794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:44.988794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:44.988794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:44.988794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:44.988794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:44.988794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:44.988794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:44.988794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:44.988794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:44.988794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:44.988794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:44.988794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:44.988794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:44.988794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:44.989756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:44.990622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:44.990622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:44.990622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:44.990622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:44.990622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:44.990622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:44.990622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:44.990622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:44.990622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 353/704] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:44.991575 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e048c-a18b-45dc-a69d-75246f5f594d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:44.996117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e048c-a18b-45dc-a69d-75246f5f594d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:44.996117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e048c-a18b-45dc-a69d-75246f5f594d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:44.997899 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 354/705] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:44.999273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68fea8-b274-44f8-8e64-1c464b2544b7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:44.999626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68fea8-b274-44f8-8e64-1c464b2544b7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:44.999902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68fea8-b274-44f8-8e64-1c464b2544b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:45.000064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e52f6-c98b-4b0f-ab47-5199ec9cb1ef None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:45.019699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8a68fea8-b274-44f8-8e64-1c464b2544b7 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:45.020936 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8a68fea8-b274-44f8-8e64-1c464b2544b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:45.050300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b95214-9240-4efb-bb26-0332373ba272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:45.050942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b95214-9240-4efb-bb26-0332373ba272 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:45.051221 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b95214-9240-4efb-bb26-0332373ba272 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:45.051699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b95214-9240-4efb-bb26-0332373ba272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:45.064709 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b95214-9240-4efb-bb26-0332373ba272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:45.066193 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b95214-9240-4efb-bb26-0332373ba272 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:45.083610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-36b95214-9240-4efb-bb26-0332373ba272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:45.083610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:45.083610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:45.083610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:45.083610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:45.083610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:45.083610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:45.083610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:45.083610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:45.083610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:45.083610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:45.083610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:45.083610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:45.083610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:45.083610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:45.083610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:45.084828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:45.085811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:45.085811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:45.085811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:45.085811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 352/706] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:45.086679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b95214-9240-4efb-bb26-0332373ba272 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:45.107662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 355/707] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:45.108874 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68fea8-b274-44f8-8e64-1c464b2544b7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:45.152586 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:45.168928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:45.219887 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b95214-9240-4efb-bb26-0332373ba272 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:45.244205 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68fea8-b274-44f8-8e64-1c464b2544b7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:45.265784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d6e06-ed9e-479e-985d-599834b77c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:45.266845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d6e06-ed9e-479e-985d-599834b77c58 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:45.267758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d6e06-ed9e-479e-985d-599834b77c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:45.268166 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d6e06-ed9e-479e-985d-599834b77c58 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:45.269277 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d6e06-ed9e-479e-985d-599834b77c58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:45.269685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d6e06-ed9e-479e-985d-599834b77c58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:45.272174 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d6e06-ed9e-479e-985d-599834b77c58 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:45.282527 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f2994-aad3-4278-9ace-061607c847a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:45.283461 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f2994-aad3-4278-9ace-061607c847a6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:45.283739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f2994-aad3-4278-9ace-061607c847a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:45.284005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f2994-aad3-4278-9ace-061607c847a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:45.284684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f2994-aad3-4278-9ace-061607c847a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:45.285035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f2994-aad3-4278-9ace-061607c847a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:45.286512 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f2994-aad3-4278-9ace-061607c847a6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:45.288305 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 353/708] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:45.290358 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcc23b-096c-4d18-b0c5-2709175bf958 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:45.290737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcc23b-096c-4d18-b0c5-2709175bf958 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:45.291107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcc23b-096c-4d18-b0c5-2709175bf958 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:45.292839 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 354/709] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:45.293616 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcc23b-096c-4d18-b0c5-2709175bf958 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:45.300727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 356/710] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:45.301969 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4616108-872e-4af7-aca3-a0d1ac5a6d8d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:45.302255 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4616108-872e-4af7-aca3-a0d1ac5a6d8d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:45.302575 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4616108-872e-4af7-aca3-a0d1ac5a6d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:45.323868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e4616108-872e-4af7-aca3-a0d1ac5a6d8d None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:45.325189 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e4616108-872e-4af7-aca3-a0d1ac5a6d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:45.402015 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 357/711] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 07:42:45.403357 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4616108-872e-4af7-aca3-a0d1ac5a6d8d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:45.438568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcc23b-096c-4d18-b0c5-2709175bf958 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:45.440784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:45.444738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:45.486272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f8bb4-b236-4972-b26c-cf0251133225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:45.486773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f8bb4-b236-4972-b26c-cf0251133225 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:45.487054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f8bb4-b236-4972-b26c-cf0251133225 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:45.487323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f8bb4-b236-4972-b26c-cf0251133225 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:45.488036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f8bb4-b236-4972-b26c-cf0251133225 None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=81d5689e9a344bafb74ca423ffd2b42a) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:45.488371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f8bb4-b236-4972-b26c-cf0251133225 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:45.489793 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f8bb4-b236-4972-b26c-cf0251133225 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:45.497416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 355/712] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:42:45 2022] GET /identity/v3/role_assignments?user.id=81d5689e9a344bafb74ca423ffd2b42a&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:45.498541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f8bb4-b236-4972-b26c-cf0251133225 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:45.518415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4616108-872e-4af7-aca3-a0d1ac5a6d8d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:45.557195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0b53b-1fba-453b-885e-084c8252b58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:45.557973 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0b53b-1fba-453b-885e-084c8252b58b None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:45.558313 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0b53b-1fba-453b-885e-084c8252b58b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:45.558657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0b53b-1fba-453b-885e-084c8252b58b None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:45.559431 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c0b53b-1fba-453b-885e-084c8252b58b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:45.561004 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c0b53b-1fba-453b-885e-084c8252b58b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:45.611581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 358/713] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:42:45 2022] POST /identity/v3/users => generated 254 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:42:45.634557 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f8bb4-b236-4972-b26c-cf0251133225 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:45.690647 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6464d-f626-490b-90eb-92069b124509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:45.691688 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6464d-f626-490b-90eb-92069b124509 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:45.692187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6464d-f626-490b-90eb-92069b124509 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:45.692644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6464d-f626-490b-90eb-92069b124509 None admin] PATH_INFO: `/v3/services/network` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:45.693453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6464d-f626-490b-90eb-92069b124509 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:45.695495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6464d-f626-490b-90eb-92069b124509 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-6ca6464d-f626-490b-90eb-92069b124509 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:45.714621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:45.716240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:45.716240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:45.716240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:45.716240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:45.716240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:45.716240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:45.716240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:42:45.716240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:42:45.716240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:42:45.716240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:42:45.716240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:45.716240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:45.716240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:45.716240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:45.716240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:45.716240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:45.717066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:45.717066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:45.717066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:45.717066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:45.717066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:45.717066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:45.717066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:45.717066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:45.717066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:45.717066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:45.717066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:42:45.717066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:42:45.717066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 07:42:45.717066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:45.720237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 356/714] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:45 2022] GET /identity/v3/services/network => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:45.725990 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0b53b-1fba-453b-885e-084c8252b58b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:45.772213 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6464d-f626-490b-90eb-92069b124509 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:45.819847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:45.877999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:45.885894 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0b53b-1fba-453b-885e-084c8252b58b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:45.913147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6464d-f626-490b-90eb-92069b124509 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:45.929595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb595e-47f8-403b-980c-14600afc6b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:45.930704 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb595e-47f8-403b-980c-14600afc6b5c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:45.931382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb595e-47f8-403b-980c-14600afc6b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:45.931945 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb595e-47f8-403b-980c-14600afc6b5c None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:45.932896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb595e-47f8-403b-980c-14600afc6b5c None admin] RBAC: Adding query filter params (name=network) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:45.933460 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb595e-47f8-403b-980c-14600afc6b5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:45.935741 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb595e-47f8-403b-980c-14600afc6b5c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:45.947691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 359/715] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 7 07:42:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:45.949652 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb595e-47f8-403b-980c-14600afc6b5c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:45.952549 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371a030-1d5a-4a4c-b95b-98e5f896de51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:45.953296 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371a030-1d5a-4a4c-b95b-98e5f896de51 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:45.953909 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371a030-1d5a-4a4c-b95b-98e5f896de51 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:45.954093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371a030-1d5a-4a4c-b95b-98e5f896de51 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:45.954783 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371a030-1d5a-4a4c-b95b-98e5f896de51 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:45.956534 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371a030-1d5a-4a4c-b95b-98e5f896de51 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-f371a030-1d5a-4a4c-b95b-98e5f896de51 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:45.966995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:45.968026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:45.968026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:45.968026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:45.968026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:45.968026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:45.968026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:45.968026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:42:45.968026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:42:45.968026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:42:45.968026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:42:45.968026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:45.968026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:45.968026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:45.968026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:45.968026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:45.968026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:45.968876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:45.968876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:45.968876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:45.968876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:45.968876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:45.968876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:45.968876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:45.968876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:45.968876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:45.968876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:45.968876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:42:45.968876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:42:45.968876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 07:42:45.968876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:45.968876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 357/716] 10.222.0.26 () {56 vars in 1129 bytes} [Wed Sep 7 07:42:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:42:45.974396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371a030-1d5a-4a4c-b95b-98e5f896de51 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:46.070850 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb595e-47f8-403b-980c-14600afc6b5c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:46.082752 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:46.091486 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371a030-1d5a-4a4c-b95b-98e5f896de51 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:46.115859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3ff21-94dd-4dbc-8a14-c81036c6aee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:46.116795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3ff21-94dd-4dbc-8a14-c81036c6aee3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:46.117285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3ff21-94dd-4dbc-8a14-c81036c6aee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:46.117740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3ff21-94dd-4dbc-8a14-c81036c6aee3 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:46.118550 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3ff21-94dd-4dbc-8a14-c81036c6aee3 None admin] RBAC: Adding query filter params (type=network) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:46.119034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3ff21-94dd-4dbc-8a14-c81036c6aee3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:46.120602 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3ff21-94dd-4dbc-8a14-c81036c6aee3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:46.132208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 360/717] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 7 07:42:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:46.135558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395ee79-87aa-4afe-a1dd-f40706644267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:46.136223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395ee79-87aa-4afe-a1dd-f40706644267 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:46.136502 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395ee79-87aa-4afe-a1dd-f40706644267 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:46.136771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395ee79-87aa-4afe-a1dd-f40706644267 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:46.137462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395ee79-87aa-4afe-a1dd-f40706644267 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:46.137766 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395ee79-87aa-4afe-a1dd-f40706644267 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:46.139249 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395ee79-87aa-4afe-a1dd-f40706644267 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:46.145446 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 358/718] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Sep 7 07:42:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:46.153291 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3ff21-94dd-4dbc-8a14-c81036c6aee3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:46.171258 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:46.274645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3ff21-94dd-4dbc-8a14-c81036c6aee3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:46.312478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68686f4a-96ba-449f-9609-836209c208d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:46.313192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68686f4a-96ba-449f-9609-836209c208d9 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:46.313718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68686f4a-96ba-449f-9609-836209c208d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:46.314223 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68686f4a-96ba-449f-9609-836209c208d9 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:46.314922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68686f4a-96ba-449f-9609-836209c208d9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:46.315232 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68686f4a-96ba-449f-9609-836209c208d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:46.316653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68686f4a-96ba-449f-9609-836209c208d9 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:46.322381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 361/719] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Sep 7 07:42:46 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:47.752621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef76c3d-d5c1-46ba-8c6d-455bc113de8c None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:47.753111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef76c3d-d5c1-46ba-8c6d-455bc113de8c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:47.753111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef76c3d-d5c1-46ba-8c6d-455bc113de8c None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:47.754688 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 359/720] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:47.760117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b3bde-6f78-466a-a4ae-c5208aadab63 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:47.760525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b3bde-6f78-466a-a4ae-c5208aadab63 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:47.760704 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b3bde-6f78-466a-a4ae-c5208aadab63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:47.779256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-628b3bde-6f78-466a-a4ae-c5208aadab63 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:47.780485 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-628b3bde-6f78-466a-a4ae-c5208aadab63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:47.870637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 362/721] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:47.967849 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c023175-9209-423e-b433-f8a915407b2e None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:47.968545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c023175-9209-423e-b433-f8a915407b2e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:47.969030 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c023175-9209-423e-b433-f8a915407b2e None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:47.970943 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 360/722] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:47.976305 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc4492-ea7a-43ec-986d-69edff2b4879 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:47.976831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc4492-ea7a-43ec-986d-69edff2b4879 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:47.977314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc4492-ea7a-43ec-986d-69edff2b4879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:47.996810 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-bddc4492-ea7a-43ec-986d-69edff2b4879 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:47.998325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bddc4492-ea7a-43ec-986d-69edff2b4879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:48.000614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:48.102392 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 363/723] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:48.210997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c023175-9209-423e-b433-f8a915407b2e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:48.346327 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c023175-9209-423e-b433-f8a915407b2e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:48.394409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4a574-a9d9-4dc1-a711-c100d97a1b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:48.395391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4a574-a9d9-4dc1-a711-c100d97a1b2b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:48.395899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4a574-a9d9-4dc1-a711-c100d97a1b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:48.396401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4a574-a9d9-4dc1-a711-c100d97a1b2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:48.407613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4a574-a9d9-4dc1-a711-c100d97a1b2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:48.413207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-82d4a574-a9d9-4dc1-a711-c100d97a1b2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:42:48.413207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:48.413207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:48.413207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:48.413207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:48.413207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:48.413207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:48.413207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:48.413207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:48.413207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:48.413207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:48.413207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:48.413207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:48.413207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:48.413207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:48.413207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:48.414120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:48.414120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:48.414120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:48.414120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:48.414120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:48.414120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:48.414120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:48.414120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:48.414120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:48.414120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:48.414120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:48.414120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:48.414120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:48.414120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:48.414120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:48.414120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:48.414942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:48.414942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:48.414942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:48.414942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:48.414942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:48.414942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:48.414942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:48.414942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:48.414942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:48.414942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:48.414942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:42:48.414942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:48.417776 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 361/724] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:42:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:48.428173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc4492-ea7a-43ec-986d-69edff2b4879 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:48.529547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4a574-a9d9-4dc1-a711-c100d97a1b2b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:48.563603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc4492-ea7a-43ec-986d-69edff2b4879 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:48.569163 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:48.603987 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e094b6-0759-4ffc-a0d0-55f3cc7e760b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:48.604630 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e094b6-0759-4ffc-a0d0-55f3cc7e760b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:48.604906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e094b6-0759-4ffc-a0d0-55f3cc7e760b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:48.605176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e094b6-0759-4ffc-a0d0-55f3cc7e760b None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:48.605830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e094b6-0759-4ffc-a0d0-55f3cc7e760b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:48.612115 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e094b6-0759-4ffc-a0d0-55f3cc7e760b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:48.612115 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e094b6-0759-4ffc-a0d0-55f3cc7e760b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:48.614311 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:48.624104 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 364/725] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:48.628923 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e094b6-0759-4ffc-a0d0-55f3cc7e760b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:48.725620 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4a574-a9d9-4dc1-a711-c100d97a1b2b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:48.781205 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328ccc6-ea2d-437e-b1ea-fe7ef00898a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:48.782404 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328ccc6-ea2d-437e-b1ea-fe7ef00898a2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:48.782808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328ccc6-ea2d-437e-b1ea-fe7ef00898a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:48.783291 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328ccc6-ea2d-437e-b1ea-fe7ef00898a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:48.814872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328ccc6-ea2d-437e-b1ea-fe7ef00898a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:48.828906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-9328ccc6-ea2d-437e-b1ea-fe7ef00898a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:42:48.828906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:48.828906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:48.828906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:48.828906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:48.828906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:48.828906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:48.828906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:48.828906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:48.828906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:48.828906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:48.828906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:48.828906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:48.828906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:48.828906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:48.828906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:48.830062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:48.830062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:48.830062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:48.830062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:48.830062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:48.830062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:48.830062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:48.830062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:48.830062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:48.830062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:48.830062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:48.830062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:48.830062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:48.830062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:48.830062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:48.830062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:48.830883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:48.830883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:48.830883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:48.830883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:48.830883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:48.830883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:48.830883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:48.830883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:48.830883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:48.830883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:48.830883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:42:48.830883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:48.833757 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 362/726] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:42:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:48.835975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328ccc6-ea2d-437e-b1ea-fe7ef00898a2 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:48.852216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e094b6-0759-4ffc-a0d0-55f3cc7e760b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:48.896749 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcf08d-0a0d-4da7-82de-2b09f3363127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:48.897381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcf08d-0a0d-4da7-82de-2b09f3363127 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:48.897659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcf08d-0a0d-4da7-82de-2b09f3363127 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:48.897931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcf08d-0a0d-4da7-82de-2b09f3363127 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:48.912980 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadcf08d-0a0d-4da7-82de-2b09f3363127 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:48.915060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadcf08d-0a0d-4da7-82de-2b09f3363127 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:48.928613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:48.930351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-eadcf08d-0a0d-4da7-82de-2b09f3363127 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:48.930351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:48.930351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:48.930351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:48.930351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:48.930351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:48.930351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:48.930351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:48.930351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:48.930351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:48.930351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:48.930351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:48.930351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:48.930351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:48.930351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:48.930351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:48.932927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:48.934319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:48.934319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:48.934319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:48.934319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 365/727] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:48.934319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c708e0-957c-4eb3-ab6d-1b528d874a20 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:48.934319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c708e0-957c-4eb3-ab6d-1b528d874a20 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:48.934319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c708e0-957c-4eb3-ab6d-1b528d874a20 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:48.939942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 366/728] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:48.939942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4634e-7082-42bb-af9b-7475c04929ea None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:48.939942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4634e-7082-42bb-af9b-7475c04929ea None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:48.939942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4634e-7082-42bb-af9b-7475c04929ea None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:48.939942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 367/729] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:48.940543 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4634e-7082-42bb-af9b-7475c04929ea None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:48.964100 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328ccc6-ea2d-437e-b1ea-fe7ef00898a2 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:48.999283 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:49.002211 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de446a14-24ed-427f-9547-27fd422e61bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:49.002858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de446a14-24ed-427f-9547-27fd422e61bb None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:49.003185 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de446a14-24ed-427f-9547-27fd422e61bb None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:49.003424 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de446a14-24ed-427f-9547-27fd422e61bb None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:49.004098 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de446a14-24ed-427f-9547-27fd422e61bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:49.004414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de446a14-24ed-427f-9547-27fd422e61bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:49.005786 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de446a14-24ed-427f-9547-27fd422e61bb None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:49.019005 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 363/730] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:49.019005 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ad654-ae78-4aa6-9c2f-0e8b17e4a8cb None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:49.019005 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ad654-ae78-4aa6-9c2f-0e8b17e4a8cb None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:49.019005 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ad654-ae78-4aa6-9c2f-0e8b17e4a8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:49.036462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3f7ad654-ae78-4aa6-9c2f-0e8b17e4a8cb None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:49.037667 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3f7ad654-ae78-4aa6-9c2f-0e8b17e4a8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:49.058991 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4634e-7082-42bb-af9b-7475c04929ea None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:49.099874 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623d5b0-e532-4d83-b4ea-28baf3ddf947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:49.100793 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623d5b0-e532-4d83-b4ea-28baf3ddf947 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:49.101080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623d5b0-e532-4d83-b4ea-28baf3ddf947 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:49.101349 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623d5b0-e532-4d83-b4ea-28baf3ddf947 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:49.102150 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623d5b0-e532-4d83-b4ea-28baf3ddf947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:49.102450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623d5b0-e532-4d83-b4ea-28baf3ddf947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:49.104421 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623d5b0-e532-4d83-b4ea-28baf3ddf947 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:49.113384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 364/731] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:49.114637 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2add8fc-52a8-4fec-87d7-60aa4ac90941 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:49.114916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2add8fc-52a8-4fec-87d7-60aa4ac90941 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:49.115189 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2add8fc-52a8-4fec-87d7-60aa4ac90941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:49.127644 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 368/732] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:49.128764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623d5b0-e532-4d83-b4ea-28baf3ddf947 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:49.139183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f2add8fc-52a8-4fec-87d7-60aa4ac90941 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:49.140486 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f2add8fc-52a8-4fec-87d7-60aa4ac90941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:49.220587 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:49.232383 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 365/733] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:49.233455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2add8fc-52a8-4fec-87d7-60aa4ac90941 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:49.247825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623d5b0-e532-4d83-b4ea-28baf3ddf947 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:49.261739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:49.283844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041873b-26de-4114-9a25-bbfe98b619a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:49.284505 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041873b-26de-4114-9a25-bbfe98b619a0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:49.284828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041873b-26de-4114-9a25-bbfe98b619a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:49.285109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041873b-26de-4114-9a25-bbfe98b619a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:49.295293 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041873b-26de-4114-9a25-bbfe98b619a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:49.296764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041873b-26de-4114-9a25-bbfe98b619a0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:49.311653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-5041873b-26de-4114-9a25-bbfe98b619a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:49.311653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:49.311653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:49.311653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:49.311653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:49.311653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:49.311653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:49.311653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:49.311653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:49.311653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:49.311653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:49.311653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:49.311653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:49.311653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:49.311653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:49.311653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:49.312598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:49.313753 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:49.313753 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:49.313753 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:49.317465 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 369/734] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:49.318718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20f0bd-8823-48b4-8af5-8767a332e422 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:49.318998 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20f0bd-8823-48b4-8af5-8767a332e422 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:49.319271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20f0bd-8823-48b4-8af5-8767a332e422 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:49.321123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 370/735] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:49.325936 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20f0bd-8823-48b4-8af5-8767a332e422 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:49.364602 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2add8fc-52a8-4fec-87d7-60aa4ac90941 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:49.408374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179d90c-b41d-400f-948e-60b723a69756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:49.409610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179d90c-b41d-400f-948e-60b723a69756 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:49.410034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179d90c-b41d-400f-948e-60b723a69756 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:49.410444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179d90c-b41d-400f-948e-60b723a69756 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:49.422413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d179d90c-b41d-400f-948e-60b723a69756 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:49.424060 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d179d90c-b41d-400f-948e-60b723a69756 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:49.434537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20f0bd-8823-48b4-8af5-8767a332e422 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:49.436161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-d179d90c-b41d-400f-948e-60b723a69756 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 07:42:49.436161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:49.436161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:49.436161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:49.436161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:49.436161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:49.436161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:49.436161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:49.436161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:49.436161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:49.436161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:49.436161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:49.436161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:49.436161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:49.436161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:49.436161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:49.437089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:49.438182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:49.438182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:49.438182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:49.438182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:49.438182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:49.438182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:49.438182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:49.438182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:49.438182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 07:42:49.438182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:49.438182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 366/736] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:42:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:49.438878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb5c0e-a7da-4179-a972-66e371d13dac None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:49.438878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb5c0e-a7da-4179-a972-66e371d13dac None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:49.440390 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb5c0e-a7da-4179-a972-66e371d13dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:49.459183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-53bb5c0e-a7da-4179-a972-66e371d13dac None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:49.460404 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-53bb5c0e-a7da-4179-a972-66e371d13dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:49.476443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7627a2-bf01-4d72-9e8e-7c30a2b0b4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:49.477045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7627a2-bf01-4d72-9e8e-7c30a2b0b4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:49.477318 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7627a2-bf01-4d72-9e8e-7c30a2b0b4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:49.477588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7627a2-bf01-4d72-9e8e-7c30a2b0b4a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:49.478217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7627a2-bf01-4d72-9e8e-7c30a2b0b4a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:49.478513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7627a2-bf01-4d72-9e8e-7c30a2b0b4a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:49.479849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7627a2-bf01-4d72-9e8e-7c30a2b0b4a2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:49.492381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 371/737] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:49.493244 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7627a2-bf01-4d72-9e8e-7c30a2b0b4a2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:49.518753 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:49.523925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:49.535903 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 367/738] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 07:42:49.536557 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb5c0e-a7da-4179-a972-66e371d13dac None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:49.608643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7627a2-bf01-4d72-9e8e-7c30a2b0b4a2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:49.640940 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb5c0e-a7da-4179-a972-66e371d13dac None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:49.643324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df0523-d702-4669-bdcb-2a7564ac09d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:49.643951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df0523-d702-4669-bdcb-2a7564ac09d8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:49.644261 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df0523-d702-4669-bdcb-2a7564ac09d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:49.644529 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df0523-d702-4669-bdcb-2a7564ac09d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:49.654752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df0523-d702-4669-bdcb-2a7564ac09d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:49.659666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-68df0523-d702-4669-bdcb-2a7564ac09d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:49.659666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:49.659666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:49.659666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:49.659666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:49.659666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:49.659666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:49.659666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:49.659666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:49.659666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:49.659666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:49.659666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:49.659666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:49.659666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:49.659666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:49.659666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:49.660622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:49.660622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:49.660622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:49.660622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:49.660622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:49.660622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:49.660622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:49.660622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:49.660622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:49.660622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:49.660622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:49.660622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:49.660622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:49.660622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:49.660622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:49.660622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:49.661447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:49.661447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:49.661447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:49.661447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:49.661447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:49.661447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:49.661447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:49.661447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:49.661447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:49.661447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:49.661447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:49.661447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:49.661447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 372/739] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:49.662127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df0523-d702-4669-bdcb-2a7564ac09d8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:49.681416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09344aec-ca78-4961-8773-e18e3a0adb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:49.682014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09344aec-ca78-4961-8773-e18e3a0adb22 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:49.682288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09344aec-ca78-4961-8773-e18e3a0adb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:49.682552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09344aec-ca78-4961-8773-e18e3a0adb22 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:49.683268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09344aec-ca78-4961-8773-e18e3a0adb22 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:49.683567 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09344aec-ca78-4961-8773-e18e3a0adb22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:49.685007 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09344aec-ca78-4961-8773-e18e3a0adb22 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:49.708710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 368/740] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Sep 7 07:42:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:49.708710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09344aec-ca78-4961-8773-e18e3a0adb22 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:49.755711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:49.762472 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:49.774344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df0523-d702-4669-bdcb-2a7564ac09d8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:49.809667 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadedadc-0af0-4041-b0ab-9a6ca018712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:49.810286 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadedadc-0af0-4041-b0ab-9a6ca018712a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:49.810564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadedadc-0af0-4041-b0ab-9a6ca018712a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:49.810835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadedadc-0af0-4041-b0ab-9a6ca018712a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:49.816586 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09344aec-ca78-4961-8773-e18e3a0adb22 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:49.818067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadedadc-0af0-4041-b0ab-9a6ca018712a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:49.819761 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadedadc-0af0-4041-b0ab-9a6ca018712a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:49.831191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-aadedadc-0af0-4041-b0ab-9a6ca018712a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:42:49.831191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:49.831191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:49.831191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:49.831191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:49.831191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:49.831191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:49.831191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:49.831191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:49.831191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:49.831191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:49.831191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:49.831191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:49.831191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:49.831191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:49.831191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:49.832266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:49.833177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:49.833177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:49.833177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:49.833177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:49.833177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:49.833177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:49.833177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:49.833177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:49.833177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:42:49.833177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:49.833177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 373/741] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 7 07:42:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:49.833756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadedadc-0af0-4041-b0ab-9a6ca018712a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:49.855655 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3dc06-4e32-47df-9582-92cdd0a12bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:49.856286 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3dc06-4e32-47df-9582-92cdd0a12bbe None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:49.856566 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3dc06-4e32-47df-9582-92cdd0a12bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:49.856839 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3dc06-4e32-47df-9582-92cdd0a12bbe None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:49.857477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3dc06-4e32-47df-9582-92cdd0a12bbe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:49.858850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3dc06-4e32-47df-9582-92cdd0a12bbe None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:49.874670 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 369/742] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 7 07:42:49 2022] POST /identity/v3/services => generated 240 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:42:49.876026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3dc06-4e32-47df-9582-92cdd0a12bbe None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:49.948955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadedadc-0af0-4041-b0ab-9a6ca018712a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:50.005182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa80f64-d7a5-4bf2-b305-07479455c5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:50.005814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa80f64-d7a5-4bf2-b305-07479455c5ab None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:50.006089 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa80f64-d7a5-4bf2-b305-07479455c5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:50.006355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa80f64-d7a5-4bf2-b305-07479455c5ab None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:50.007018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa80f64-d7a5-4bf2-b305-07479455c5ab None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:50.007370 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa80f64-d7a5-4bf2-b305-07479455c5ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:50.008771 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa80f64-d7a5-4bf2-b305-07479455c5ab None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:50.009117 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3dc06-4e32-47df-9582-92cdd0a12bbe None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:50.012665 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:50.023646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 374/743] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:50.024618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa80f64-d7a5-4bf2-b305-07479455c5ab None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:50.027475 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:50.050394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776e31c-e856-4532-97da-95a6dbb4d569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:50.051011 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776e31c-e856-4532-97da-95a6dbb4d569 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:50.051360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776e31c-e856-4532-97da-95a6dbb4d569 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:50.051646 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776e31c-e856-4532-97da-95a6dbb4d569 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:50.126628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e776e31c-e856-4532-97da-95a6dbb4d569 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:50.128117 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e776e31c-e856-4532-97da-95a6dbb4d569 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:50.142265 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa80f64-d7a5-4bf2-b305-07479455c5ab None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:50.178815 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03203e2c-d5cd-4f0c-81a2-2d3fd6085e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:50.179426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03203e2c-d5cd-4f0c-81a2-2d3fd6085e09 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:50.179699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03203e2c-d5cd-4f0c-81a2-2d3fd6085e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:50.179971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03203e2c-d5cd-4f0c-81a2-2d3fd6085e09 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:50.180651 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03203e2c-d5cd-4f0c-81a2-2d3fd6085e09 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:50.182007 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03203e2c-d5cd-4f0c-81a2-2d3fd6085e09 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:50.204117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 375/744] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 7 07:42:50 2022] POST /identity/v3/services => generated 251 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:42:50.204117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03203e2c-d5cd-4f0c-81a2-2d3fd6085e09 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:50.261554 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 370/745] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:49 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 07:42:50.262448 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776e31c-e856-4532-97da-95a6dbb4d569 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:50.268272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:50.275608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:50.332732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03203e2c-d5cd-4f0c-81a2-2d3fd6085e09 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:50.383052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefb24c-a503-4b8b-a5ab-8dfc1ec02d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:50.383854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefb24c-a503-4b8b-a5ab-8dfc1ec02d3d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:50.384175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefb24c-a503-4b8b-a5ab-8dfc1ec02d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:50.384452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefb24c-a503-4b8b-a5ab-8dfc1ec02d3d None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:50.385189 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefb24c-a503-4b8b-a5ab-8dfc1ec02d3d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:50.385499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefb24c-a503-4b8b-a5ab-8dfc1ec02d3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:50.389715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefb24c-a503-4b8b-a5ab-8dfc1ec02d3d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:50.406362 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776e31c-e856-4532-97da-95a6dbb4d569 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:50.409004 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 376/746] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 7 07:42:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:50.409862 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefb24c-a503-4b8b-a5ab-8dfc1ec02d3d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:50.450162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ddae5-6de5-49e7-a451-d5e3bd74031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:50.451187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ddae5-6de5-49e7-a451-d5e3bd74031e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:50.451667 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ddae5-6de5-49e7-a451-d5e3bd74031e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:50.452202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ddae5-6de5-49e7-a451-d5e3bd74031e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:50.464737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ddae5-6de5-49e7-a451-d5e3bd74031e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:50.467529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ddae5-6de5-49e7-a451-d5e3bd74031e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:50.487096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-959ddae5-6de5-49e7-a451-d5e3bd74031e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:50.487096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:50.487096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:50.487096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:50.487096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:50.487096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:50.487096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:50.487096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:50.487096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:50.487096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:50.487096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:50.487096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:50.487096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:50.487096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:50.487096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:50.487096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:50.488097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:50.489550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:50.489550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:50.489550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:50.489550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 371/747] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:50.489550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ddae5-6de5-49e7-a451-d5e3bd74031e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:50.534455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefb24c-a503-4b8b-a5ab-8dfc1ec02d3d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:50.557741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:50.561505 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:50.577696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586768ec-8220-4ad0-b660-8d1c9e2e49a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:50.578672 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586768ec-8220-4ad0-b660-8d1c9e2e49a1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:50.579112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586768ec-8220-4ad0-b660-8d1c9e2e49a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:50.579697 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586768ec-8220-4ad0-b660-8d1c9e2e49a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:50.590597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586768ec-8220-4ad0-b660-8d1c9e2e49a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:50.592385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586768ec-8220-4ad0-b660-8d1c9e2e49a1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:50.608613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-586768ec-8220-4ad0-b660-8d1c9e2e49a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:50.608613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:50.608613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:50.608613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:50.608613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:50.608613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:50.608613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:50.608613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:50.608613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:50.608613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:50.608613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:50.608613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:50.608613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:50.608613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:50.608613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:50.608613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:50.609594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:50.610789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:50.610789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:50.610789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:50.612979 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 377/748] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:50.613986 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586768ec-8220-4ad0-b660-8d1c9e2e49a1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:50.618913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ddae5-6de5-49e7-a451-d5e3bd74031e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:50.657354 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2d54e-6dd8-4ff4-9032-10e51f4a65f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:50.658520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2d54e-6dd8-4ff4-9032-10e51f4a65f3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:50.659071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2d54e-6dd8-4ff4-9032-10e51f4a65f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:50.659538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2d54e-6dd8-4ff4-9032-10e51f4a65f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:50.676232 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2d54e-6dd8-4ff4-9032-10e51f4a65f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:50.678038 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2d54e-6dd8-4ff4-9032-10e51f4a65f3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:50.698936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-78f2d54e-6dd8-4ff4-9032-10e51f4a65f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:50.698936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:50.698936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:50.698936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:50.698936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:50.698936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:50.698936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:50.698936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:50.698936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:50.698936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:50.698936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:50.698936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:50.698936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:50.698936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:50.698936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:50.698936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:50.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:50.700825 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:50.700825 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:50.700825 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:50.707346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 372/749] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:50.708564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2d54e-6dd8-4ff4-9032-10e51f4a65f3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:50.742049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586768ec-8220-4ad0-b660-8d1c9e2e49a1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:50.779679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224299a-c16c-4858-8483-4752040f758d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:50.780308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224299a-c16c-4858-8483-4752040f758d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:50.780646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224299a-c16c-4858-8483-4752040f758d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:50.780921 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224299a-c16c-4858-8483-4752040f758d None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:50.781573 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224299a-c16c-4858-8483-4752040f758d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:50.781926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224299a-c16c-4858-8483-4752040f758d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:50.783414 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224299a-c16c-4858-8483-4752040f758d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:50.796921 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 378/750] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:50.798350 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224299a-c16c-4858-8483-4752040f758d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:50.831375 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:50.850274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2d54e-6dd8-4ff4-9032-10e51f4a65f3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:50.865470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:50.899418 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7277ef7-6fda-481c-b887-400fe575f1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:50.900777 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7277ef7-6fda-481c-b887-400fe575f1aa None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:50.901455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7277ef7-6fda-481c-b887-400fe575f1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:50.901870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7277ef7-6fda-481c-b887-400fe575f1aa None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:50.902880 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7277ef7-6fda-481c-b887-400fe575f1aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:50.903343 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7277ef7-6fda-481c-b887-400fe575f1aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:50.905520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7277ef7-6fda-481c-b887-400fe575f1aa None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:50.912818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224299a-c16c-4858-8483-4752040f758d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:50.922795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 373/751] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:50.924463 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7277ef7-6fda-481c-b887-400fe575f1aa None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:50.954110 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f37001-c45c-4bc9-85de-12602ff92706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:50.954881 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f37001-c45c-4bc9-85de-12602ff92706 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:50.955249 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f37001-c45c-4bc9-85de-12602ff92706 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:50.955612 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f37001-c45c-4bc9-85de-12602ff92706 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:50.956414 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f37001-c45c-4bc9-85de-12602ff92706 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:50.956866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f37001-c45c-4bc9-85de-12602ff92706 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:50.958533 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f37001-c45c-4bc9-85de-12602ff92706 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:50.975993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 379/752] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:50.977056 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f37001-c45c-4bc9-85de-12602ff92706 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:51.052796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7277ef7-6fda-481c-b887-400fe575f1aa None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:51.095772 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:51.102118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4014e9e-bcbf-4810-ab3b-535459536662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:51.102837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4014e9e-bcbf-4810-ab3b-535459536662 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:51.103210 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4014e9e-bcbf-4810-ab3b-535459536662 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:51.103604 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4014e9e-bcbf-4810-ab3b-535459536662 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:51.111629 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4014e9e-bcbf-4810-ab3b-535459536662 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:51.113198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4014e9e-bcbf-4810-ab3b-535459536662 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:51.114410 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f37001-c45c-4bc9-85de-12602ff92706 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:51.125410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-d4014e9e-bcbf-4810-ab3b-535459536662 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 07:42:51.125410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:51.125410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:51.125410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:51.125410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:51.125410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:51.125410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:51.125410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:51.125410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:51.125410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:51.125410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:51.125410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:51.125410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:51.125410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:51.125410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:51.125410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:51.126541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:51.127681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:51.127681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:51.127681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:51.127681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:51.127681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:51.127681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:51.127681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:51.127681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:51.127681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 07:42:51.127681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:51.129491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 374/753] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 7 07:42:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:51.130563 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4014e9e-bcbf-4810-ab3b-535459536662 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:51.163718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbb743-27aa-4fff-be24-951cba35f320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:51.164712 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbb743-27aa-4fff-be24-951cba35f320 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:51.165023 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:51.165141 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbb743-27aa-4fff-be24-951cba35f320 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:51.165585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbb743-27aa-4fff-be24-951cba35f320 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:51.177291 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edbb743-27aa-4fff-be24-951cba35f320 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:51.179307 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edbb743-27aa-4fff-be24-951cba35f320 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:51.197821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-8edbb743-27aa-4fff-be24-951cba35f320 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:51.197821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:51.197821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:51.197821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:51.197821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:51.197821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:51.197821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:51.197821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:51.197821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:51.197821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:51.197821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:51.197821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:51.197821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:51.197821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:51.197821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:51.198814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:51.199686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:51.199686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:51.199686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:51.199686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:51.199686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:51.199686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:51.199686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:51.199686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:51.200132 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 380/754] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:51.201505 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbb743-27aa-4fff-be24-951cba35f320 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:51.240550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4014e9e-bcbf-4810-ab3b-535459536662 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:51.275797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b496b-89ca-4e72-b9b2-e5bdc3a6662f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:51.276552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b496b-89ca-4e72-b9b2-e5bdc3a6662f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:51.276897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b496b-89ca-4e72-b9b2-e5bdc3a6662f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:51.277258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b496b-89ca-4e72-b9b2-e5bdc3a6662f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:51.284782 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b496b-89ca-4e72-b9b2-e5bdc3a6662f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:51.286392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b496b-89ca-4e72-b9b2-e5bdc3a6662f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:51.298389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-e74b496b-89ca-4e72-b9b2-e5bdc3a6662f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:51.298389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:51.298389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:51.298389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:51.298389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:51.298389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:51.298389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:51.298389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:51.298389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:51.298389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:51.298389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:51.298389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:51.298389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:51.298389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:51.298389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:51.299268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:51.300175 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:51.300175 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:51.300175 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:51.300175 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:51.300175 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:51.300175 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:51.300175 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:51.300175 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:51.304414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 375/755] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:51.305260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b496b-89ca-4e72-b9b2-e5bdc3a6662f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:51.323892 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbb743-27aa-4fff-be24-951cba35f320 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:51.368525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fb798-79e2-4e6c-98f5-a279dc870c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:51.369328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fb798-79e2-4e6c-98f5-a279dc870c10 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:51.369772 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fb798-79e2-4e6c-98f5-a279dc870c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:51.370207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fb798-79e2-4e6c-98f5-a279dc870c10 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:51.371869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042fb798-79e2-4e6c-98f5-a279dc870c10 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:51.372406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042fb798-79e2-4e6c-98f5-a279dc870c10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:51.374433 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042fb798-79e2-4e6c-98f5-a279dc870c10 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:51.402406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 381/756] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Sep 7 07:42:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:51.404129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fb798-79e2-4e6c-98f5-a279dc870c10 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:51.405652 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:51.422092 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b496b-89ca-4e72-b9b2-e5bdc3a6662f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:51.440625 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:51.458571 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7869b-a534-411f-aeaa-3fb52c003917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:51.459062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7869b-a534-411f-aeaa-3fb52c003917 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:51.459413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7869b-a534-411f-aeaa-3fb52c003917 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:51.459757 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7869b-a534-411f-aeaa-3fb52c003917 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:51.460627 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7869b-a534-411f-aeaa-3fb52c003917 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:51.461020 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7869b-a534-411f-aeaa-3fb52c003917 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:51.462569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7869b-a534-411f-aeaa-3fb52c003917 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:51.476035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 376/757] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:51.476771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7869b-a534-411f-aeaa-3fb52c003917 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:51.536826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fb798-79e2-4e6c-98f5-a279dc870c10 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:51.581138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7869b-a534-411f-aeaa-3fb52c003917 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:51.581730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c162101-e352-4903-821b-ab9f2e1fafd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:51.582820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c162101-e352-4903-821b-ab9f2e1fafd7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:51.583535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c162101-e352-4903-821b-ab9f2e1fafd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:51.584187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c162101-e352-4903-821b-ab9f2e1fafd7 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:51.585731 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c162101-e352-4903-821b-ab9f2e1fafd7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:51.586414 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c162101-e352-4903-821b-ab9f2e1fafd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:51.588568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c162101-e352-4903-821b-ab9f2e1fafd7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:51.604726 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 382/758] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:51.606011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c162101-e352-4903-821b-ab9f2e1fafd7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:51.624315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4153b5e-1501-4440-8ad5-7348c8eef1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:51.624917 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4153b5e-1501-4440-8ad5-7348c8eef1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:51.625195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4153b5e-1501-4440-8ad5-7348c8eef1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:51.625758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4153b5e-1501-4440-8ad5-7348c8eef1a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:51.645212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:51.686826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:51.698985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4153b5e-1501-4440-8ad5-7348c8eef1a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:51.700507 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4153b5e-1501-4440-8ad5-7348c8eef1a0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:51.729043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c162101-e352-4903-821b-ab9f2e1fafd7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:51.766764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fac8fe-c7f8-4151-9121-7ba2eb767639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:51.767826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fac8fe-c7f8-4151-9121-7ba2eb767639 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:51.768344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fac8fe-c7f8-4151-9121-7ba2eb767639 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:51.768901 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fac8fe-c7f8-4151-9121-7ba2eb767639 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:51.769822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fac8fe-c7f8-4151-9121-7ba2eb767639 None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=3ff3bf8da732489b8b3a8fbe5aa22f21) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:51.770458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fac8fe-c7f8-4151-9121-7ba2eb767639 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:51.772289 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fac8fe-c7f8-4151-9121-7ba2eb767639 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:51.778642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 383/759] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:42:51 2022] GET /identity/v3/role_assignments?user.id=3ff3bf8da732489b8b3a8fbe5aa22f21&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:51.779928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fac8fe-c7f8-4151-9121-7ba2eb767639 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:51.805677 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 377/760] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:51 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:42:51.811417 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4153b5e-1501-4440-8ad5-7348c8eef1a0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:51.866346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:51.901766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fac8fe-c7f8-4151-9121-7ba2eb767639 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:51.937695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:51.946557 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e646dc-111e-4c86-a7fe-0729fba4ddbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:51.947541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e646dc-111e-4c86-a7fe-0729fba4ddbe None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:51.948049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e646dc-111e-4c86-a7fe-0729fba4ddbe None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:51.948571 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e646dc-111e-4c86-a7fe-0729fba4ddbe None admin] PATH_INFO: `/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/81d5689e9a344bafb74ca423ffd2b42a/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:51.950867 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e646dc-111e-4c86-a7fe-0729fba4ddbe None admin] RBAC: Authorizing `identity:create_grant(project_id=9a949b959e584474aafb4a05ebbabcc4, user_id=81d5689e9a344bafb74ca423ffd2b42a, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:51.952600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e646dc-111e-4c86-a7fe-0729fba4ddbe None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:51.962737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4153b5e-1501-4440-8ad5-7348c8eef1a0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:51.978656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 384/761] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:42:51 2022] PUT /identity/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/81d5689e9a344bafb74ca423ffd2b42a/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:42:52.016914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb632c-83b4-421a-896e-9dcb31b0bef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:52.017935 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb632c-83b4-421a-896e-9dcb31b0bef1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:52.018512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb632c-83b4-421a-896e-9dcb31b0bef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:52.019016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb632c-83b4-421a-896e-9dcb31b0bef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:52.031786 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb632c-83b4-421a-896e-9dcb31b0bef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:52.033920 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb632c-83b4-421a-896e-9dcb31b0bef1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:52.051065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-6aeb632c-83b4-421a-896e-9dcb31b0bef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:52.051065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:52.051065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:52.051065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:52.051065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:52.051065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:52.051065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:52.051065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:52.051065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:52.051065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:52.051065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:52.051065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:52.051065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:52.051065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:52.051065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:52.051065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:52.052138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:52.053186 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:52.053186 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:52.053186 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:52.055063 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 378/762] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:42:52.061219 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e646dc-111e-4c86-a7fe-0729fba4ddbe None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:52.200028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e23a6b-ba02-4ae9-8963-47d04417ee89 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:52.200456 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e23a6b-ba02-4ae9-8963-47d04417ee89 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:52.200811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e23a6b-ba02-4ae9-8963-47d04417ee89 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:52.202229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 379/763] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:42:52.208574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5e0f4-2c03-4dc2-af80-43ebbbbf4e2f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:52.208930 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5e0f4-2c03-4dc2-af80-43ebbbbf4e2f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:52.209278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5e0f4-2c03-4dc2-af80-43ebbbbf4e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:52.220971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e646dc-111e-4c86-a7fe-0729fba4ddbe None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:52.232367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-11b5e0f4-2c03-4dc2-af80-43ebbbbf4e2f None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:52.233652 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-11b5e0f4-2c03-4dc2-af80-43ebbbbf4e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:52.263001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d4132-2144-48b1-95a1-7e5d4278a245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:52.263903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d4132-2144-48b1-95a1-7e5d4278a245 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:52.264397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d4132-2144-48b1-95a1-7e5d4278a245 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:52.264868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d4132-2144-48b1-95a1-7e5d4278a245 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:52.265503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d4132-2144-48b1-95a1-7e5d4278a245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:52.265803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d4132-2144-48b1-95a1-7e5d4278a245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:52.267747 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d4132-2144-48b1-95a1-7e5d4278a245 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:52.283154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 385/764] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:52.290771 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d4132-2144-48b1-95a1-7e5d4278a245 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:52.358302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 380/765] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:42:52.361052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:52.419226 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d4132-2144-48b1-95a1-7e5d4278a245 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:52.459621 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82612e04-2004-4f30-bcfb-f60eb3747ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:52.460213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82612e04-2004-4f30-bcfb-f60eb3747ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:52.460492 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82612e04-2004-4f30-bcfb-f60eb3747ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:52.460762 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82612e04-2004-4f30-bcfb-f60eb3747ac6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:52.468103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82612e04-2004-4f30-bcfb-f60eb3747ac6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:52.470231 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82612e04-2004-4f30-bcfb-f60eb3747ac6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:52.486135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-82612e04-2004-4f30-bcfb-f60eb3747ac6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:52.486135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:52.486135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:52.486135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:52.486135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:52.486135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:52.486135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:52.486135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:52.486135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:52.486135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:52.486135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:52.486135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:52.486135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:52.486135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:52.486135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:52.487025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:52.487897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:52.487897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:52.487897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:52.487897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:52.487897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:52.487897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:52.487897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:52.487897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:52.489803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 386/766] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:52.494793 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5e0f4-2c03-4dc2-af80-43ebbbbf4e2f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:52.556458 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:52.642797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5e0f4-2c03-4dc2-af80-43ebbbbf4e2f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:52.678271 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbc3df-161e-4db4-9c3b-81e2fff020a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:52.679264 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbc3df-161e-4db4-9c3b-81e2fff020a3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:52.679744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbc3df-161e-4db4-9c3b-81e2fff020a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:52.680720 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbc3df-161e-4db4-9c3b-81e2fff020a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:52.681383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82612e04-2004-4f30-bcfb-f60eb3747ac6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:52.682749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adbc3df-161e-4db4-9c3b-81e2fff020a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:52.683591 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adbc3df-161e-4db4-9c3b-81e2fff020a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:52.686946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adbc3df-161e-4db4-9c3b-81e2fff020a3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:52.702341 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 381/767] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:52.709173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbc3df-161e-4db4-9c3b-81e2fff020a3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:52.813786 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:52.822145 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbc3df-161e-4db4-9c3b-81e2fff020a3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:52.848932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82612e04-2004-4f30-bcfb-f60eb3747ac6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:52.859578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d50a8-28ba-498c-a3b9-813c99ebd86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:52.860797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d50a8-28ba-498c-a3b9-813c99ebd86f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:52.861280 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d50a8-28ba-498c-a3b9-813c99ebd86f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:52.861716 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:52.861983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d50a8-28ba-498c-a3b9-813c99ebd86f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:52.862895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d50a8-28ba-498c-a3b9-813c99ebd86f None admin] RBAC: Adding query filter params (role.id=e7960e6d957f4a1fa6547565db026f24, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=617f6479b90945d3b5a63325839afb15) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:52.863423 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d50a8-28ba-498c-a3b9-813c99ebd86f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:52.865094 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d50a8-28ba-498c-a3b9-813c99ebd86f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:52.871460 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 382/768] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:42:52 2022] GET /identity/v3/role_assignments?user.id=617f6479b90945d3b5a63325839afb15&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=e7960e6d957f4a1fa6547565db026f24 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:52.881803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa0941-880c-4c07-8076-d730c1b521f7 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:52.882386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa0941-880c-4c07-8076-d730c1b521f7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:52.882862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa0941-880c-4c07-8076-d730c1b521f7 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:52.885309 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 383/769] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:52.891461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4bb53-8988-4c18-8f63-e65c6cee6e9a None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:52.892116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4bb53-8988-4c18-8f63-e65c6cee6e9a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:52.892324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8df00-1f8d-4b6e-8cbc-5473ff407e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:52.892825 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4bb53-8988-4c18-8f63-e65c6cee6e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:52.892996 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8df00-1f8d-4b6e-8cbc-5473ff407e5c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:52.893250 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8df00-1f8d-4b6e-8cbc-5473ff407e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:52.893527 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8df00-1f8d-4b6e-8cbc-5473ff407e5c None admin] PATH_INFO: `/v3/services/network` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:52.894117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8df00-1f8d-4b6e-8cbc-5473ff407e5c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:52.895481 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8df00-1f8d-4b6e-8cbc-5473ff407e5c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-93b8df00-1f8d-4b6e-8cbc-5473ff407e5c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:52.907926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:52.909005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:52.909005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:52.909005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:52.909005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:52.909005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:52.909005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:52.909005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:42:52.909005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:42:52.909005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:42:52.909005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:42:52.909005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:52.909005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:52.909005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:52.909005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:52.909005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:52.909005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:52.909859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:52.909859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:52.909859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:52.909859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:52.909859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:52.909859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:52.909859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:52.909859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:52.909859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:52.909859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:52.909859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:42:52.909859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:42:52.909859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 07:42:52.909859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:52.909859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 387/770] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:52 2022] GET /identity/v3/services/network => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:42:52.915966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8df00-1f8d-4b6e-8cbc-5473ff407e5c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:52.933398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-8ce4bb53-8988-4c18-8f63-e65c6cee6e9a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:52.934992 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8ce4bb53-8988-4c18-8f63-e65c6cee6e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:53.065039 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8df00-1f8d-4b6e-8cbc-5473ff407e5c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:53.093626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 384/771] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:42:53.123797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39dc498-18fb-40ee-8cb6-4ed9270c567e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:53.124769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dc498-18fb-40ee-8cb6-4ed9270c567e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:53.125334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dc498-18fb-40ee-8cb6-4ed9270c567e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:53.126301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dc498-18fb-40ee-8cb6-4ed9270c567e None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:53.127123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39dc498-18fb-40ee-8cb6-4ed9270c567e None admin] RBAC: Adding query filter params (name=network) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:53.127621 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39dc498-18fb-40ee-8cb6-4ed9270c567e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:53.129273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39dc498-18fb-40ee-8cb6-4ed9270c567e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:53.134513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 388/772] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 7 07:42:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:53.139730 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4bb53-8988-4c18-8f63-e65c6cee6e9a None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:53.237927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:53.247590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4bb53-8988-4c18-8f63-e65c6cee6e9a None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:53.283796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255e382-203a-4a8e-bb00-29722d91a57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:53.284809 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255e382-203a-4a8e-bb00-29722d91a57a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:53.285287 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255e382-203a-4a8e-bb00-29722d91a57a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:53.285875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255e382-203a-4a8e-bb00-29722d91a57a None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:53.286694 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255e382-203a-4a8e-bb00-29722d91a57a None admin] RBAC: Adding query filter params (type=network) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:53.287210 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255e382-203a-4a8e-bb00-29722d91a57a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:53.288801 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255e382-203a-4a8e-bb00-29722d91a57a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:53.295388 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 385/773] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 7 07:42:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:53.305180 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39dc498-18fb-40ee-8cb6-4ed9270c567e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:53.356851 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:53.411298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255e382-203a-4a8e-bb00-29722d91a57a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:53.441060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39dc498-18fb-40ee-8cb6-4ed9270c567e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:53.614946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255e382-203a-4a8e-bb00-29722d91a57a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:53.633383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9fdc9-047d-4513-94e6-814310a7fd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:53.634262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9fdc9-047d-4513-94e6-814310a7fd02 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:53.634687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9fdc9-047d-4513-94e6-814310a7fd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:53.635118 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9fdc9-047d-4513-94e6-814310a7fd02 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:53.635961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9fdc9-047d-4513-94e6-814310a7fd02 None admin] RBAC: Adding query filter params (interface=public, service_id=561c2a0b3f5c4ec1a8b72cbc64453696, region_id=RegionOne) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:53.636436 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9fdc9-047d-4513-94e6-814310a7fd02 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:53.637949 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9fdc9-047d-4513-94e6-814310a7fd02 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:53.652509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0099da-56c3-4b57-bbd6-460ceefcb371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:53.652954 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0099da-56c3-4b57-bbd6-460ceefcb371 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:53.653323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0099da-56c3-4b57-bbd6-460ceefcb371 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:53.653670 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0099da-56c3-4b57-bbd6-460ceefcb371 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:53.654319 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0099da-56c3-4b57-bbd6-460ceefcb371 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:53.655749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0099da-56c3-4b57-bbd6-460ceefcb371 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:53.658900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 389/774] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Sep 7 07:42:53 2022] GET /identity/v3/endpoints?service_id=561c2a0b3f5c4ec1a8b72cbc64453696&interface=public®ion_id=RegionOne => generated 194 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:53.664371 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9fdc9-047d-4513-94e6-814310a7fd02 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-ea0099da-56c3-4b57-bbd6-460ceefcb371 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:53.670365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:53.671394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:53.671394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:53.671394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:53.671394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:53.671394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:53.671394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:53.671394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:42:53.671394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:42:53.671394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:42:53.671394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:42:53.671394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:53.671394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:53.671394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:53.671394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:53.671394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:53.671394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:53.672758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:53.672758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:53.672758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:53.672758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:53.672758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:53.672758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:53.672758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:53.672758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:53.672758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:53.672758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:53.672758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:42:53.672758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:42:53.672758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 07:42:53.672758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:53.672758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 386/775] 10.222.0.26 () {56 vars in 1129 bytes} [Wed Sep 7 07:42:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:42:53.677906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0099da-56c3-4b57-bbd6-460ceefcb371 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:53.696041 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:53.851563 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9fdc9-047d-4513-94e6-814310a7fd02 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:53.854868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0099da-56c3-4b57-bbd6-460ceefcb371 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:53.856515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:53.917065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5533d6-c53f-4dfe-bd2d-b13d8ddd9cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:53.917733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5533d6-c53f-4dfe-bd2d-b13d8ddd9cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:53.917961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5533d6-c53f-4dfe-bd2d-b13d8ddd9cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:53.918242 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5533d6-c53f-4dfe-bd2d-b13d8ddd9cb6 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:53.918879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5533d6-c53f-4dfe-bd2d-b13d8ddd9cb6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:53.919190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5533d6-c53f-4dfe-bd2d-b13d8ddd9cb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:53.920699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5533d6-c53f-4dfe-bd2d-b13d8ddd9cb6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:53.927341 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 387/776] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Sep 7 07:42:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:53.933055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5533d6-c53f-4dfe-bd2d-b13d8ddd9cb6 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:53.938821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5cb88-2832-47c6-be9b-510be08cbfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:53.939472 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5cb88-2832-47c6-be9b-510be08cbfc2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:53.939750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5cb88-2832-47c6-be9b-510be08cbfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:53.940156 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5cb88-2832-47c6-be9b-510be08cbfc2 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:53.940703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5cb88-2832-47c6-be9b-510be08cbfc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:53.942057 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5cb88-2832-47c6-be9b-510be08cbfc2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:53.956984 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 390/777] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 7 07:42:53 2022] GET /identity/v3/services => generated 772 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:42:54.054270 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:54.082299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5533d6-c53f-4dfe-bd2d-b13d8ddd9cb6 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:54.130728 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011fc68-908c-436d-8162-a2083cb9a559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:54.131349 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011fc68-908c-436d-8162-a2083cb9a559 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:54.131626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011fc68-908c-436d-8162-a2083cb9a559 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:54.131896 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011fc68-908c-436d-8162-a2083cb9a559 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:54.132551 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011fc68-908c-436d-8162-a2083cb9a559 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:54.132854 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011fc68-908c-436d-8162-a2083cb9a559 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:54.134290 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011fc68-908c-436d-8162-a2083cb9a559 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:54.145033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 388/778] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Sep 7 07:42:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:54.151429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5cb88-2832-47c6-be9b-510be08cbfc2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:54.273640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5cb88-2832-47c6-be9b-510be08cbfc2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:54.320034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f644399-1ddb-450b-b8c9-48bb7a669d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:54.320656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f644399-1ddb-450b-b8c9-48bb7a669d72 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:54.320926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f644399-1ddb-450b-b8c9-48bb7a669d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:54.321193 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f644399-1ddb-450b-b8c9-48bb7a669d72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:54.321883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f644399-1ddb-450b-b8c9-48bb7a669d72 None admin] RBAC: Adding query filter params (interface=public, service_id=19bc9d1b8785461fbe0c8de297a14812, region_id=RegionOne) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:54.322184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f644399-1ddb-450b-b8c9-48bb7a669d72 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:54.323534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f644399-1ddb-450b-b8c9-48bb7a669d72 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:54.329755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 391/779] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Sep 7 07:42:54 2022] GET /identity/v3/endpoints?service_id=19bc9d1b8785461fbe0c8de297a14812&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:54.341183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011fc68-908c-436d-8162-a2083cb9a559 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:54.475214 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011fc68-908c-436d-8162-a2083cb9a559 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:54.516566 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ff4d4-dba8-4750-9905-eb8ec52b53e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:54.517290 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ff4d4-dba8-4750-9905-eb8ec52b53e2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:54.517564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ff4d4-dba8-4750-9905-eb8ec52b53e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:54.517836 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ff4d4-dba8-4750-9905-eb8ec52b53e2 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:54.518453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ff4d4-dba8-4750-9905-eb8ec52b53e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:54.519888 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ff4d4-dba8-4750-9905-eb8ec52b53e2 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:54.526040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 389/780] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 7 07:42:54 2022] GET /identity/v3/services => generated 772 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:54.638079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea3892-57ce-4028-bbee-5abd3dfebb9c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:54.638414 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea3892-57ce-4028-bbee-5abd3dfebb9c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:54.638630 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea3892-57ce-4028-bbee-5abd3dfebb9c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:54.640403 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 392/781] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:54.658418 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480b966-63da-4899-9b25-909e1d0a94f6 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:54.658777 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480b966-63da-4899-9b25-909e1d0a94f6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:54.658969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480b966-63da-4899-9b25-909e1d0a94f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:54.671135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:54.684623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3480b966-63da-4899-9b25-909e1d0a94f6 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:54.685834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3480b966-63da-4899-9b25-909e1d0a94f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:54.718850 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e110d4c-e95f-4623-ba3c-26682de4dfae None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:54.719245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e110d4c-e95f-4623-ba3c-26682de4dfae None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:54.719503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e110d4c-e95f-4623-ba3c-26682de4dfae None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:54.721243 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 393/782] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:54.727089 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba745e-8ca9-46eb-8e57-9ba4a7017513 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:54.727411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba745e-8ca9-46eb-8e57-9ba4a7017513 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:54.727656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba745e-8ca9-46eb-8e57-9ba4a7017513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:54.761318 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b1ba745e-8ca9-46eb-8e57-9ba4a7017513 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:54.762888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b1ba745e-8ca9-46eb-8e57-9ba4a7017513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:54.782243 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 390/783] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 07:42:54.808944 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:54.863837 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 394/784] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 07:42:55.066037 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480b966-63da-4899-9b25-909e1d0a94f6 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:55.183239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480b966-63da-4899-9b25-909e1d0a94f6 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:55.227593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5080d-4fa4-458d-8a50-3da9abfcf8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:55.228252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5080d-4fa4-458d-8a50-3da9abfcf8ab None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:55.228531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5080d-4fa4-458d-8a50-3da9abfcf8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:55.228797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5080d-4fa4-458d-8a50-3da9abfcf8ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:55.230802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:55.245303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5080d-4fa4-458d-8a50-3da9abfcf8ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:55.250124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-f3e5080d-4fa4-458d-8a50-3da9abfcf8ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:42:55.250124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:55.250124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:55.250124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:55.250124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:55.250124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:55.250124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:55.250124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:55.250124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:55.250124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:55.250124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:55.250124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:55.250124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:55.250124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:55.250124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:55.250124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:55.251203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:55.251203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:55.251203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:55.251203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:55.251203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:55.251203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:55.251203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:55.251203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:55.251203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:55.251203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:55.251203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:55.251203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:55.251203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:55.251203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:55.251203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:55.251203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:55.252578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:55.252578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:55.252578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:55.252578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:55.252578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:55.252578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:55.252578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:55.252578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:55.252578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:55.252578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:55.252578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:42:55.252578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:55.252578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 391/785] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:42:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:55.259380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba745e-8ca9-46eb-8e57-9ba4a7017513 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:55.281955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5080d-4fa4-458d-8a50-3da9abfcf8ab None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:55.454903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba745e-8ca9-46eb-8e57-9ba4a7017513 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:55.482181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5080d-4fa4-458d-8a50-3da9abfcf8ab None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:55.535930 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d9de1-30b5-4f99-b22e-4e536ebe4bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:55.536574 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d9de1-30b5-4f99-b22e-4e536ebe4bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:55.537434 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d9de1-30b5-4f99-b22e-4e536ebe4bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:55.537709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d9de1-30b5-4f99-b22e-4e536ebe4bc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:55.538381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d9de1-30b5-4f99-b22e-4e536ebe4bc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:55.538684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d9de1-30b5-4f99-b22e-4e536ebe4bc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:55.540043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d9de1-30b5-4f99-b22e-4e536ebe4bc2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:55.541960 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85cddb-9a52-48e0-bcdd-f243bbd6a11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:55.542588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85cddb-9a52-48e0-bcdd-f243bbd6a11e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:55.542864 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85cddb-9a52-48e0-bcdd-f243bbd6a11e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:55.543126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85cddb-9a52-48e0-bcdd-f243bbd6a11e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:55.561423 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 395/786] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:55.561888 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85cddb-9a52-48e0-bcdd-f243bbd6a11e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:55.568022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d9de1-30b5-4f99-b22e-4e536ebe4bc2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:55.571823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-0c85cddb-9a52-48e0-bcdd-f243bbd6a11e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:42:55.571823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:55.571823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:55.571823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:55.571823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:55.571823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:55.571823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:55.571823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:55.571823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:55.571823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:55.571823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:55.571823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:55.571823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:55.571823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:55.571823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:55.571823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:55.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:55.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:55.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:55.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:55.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:55.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:55.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:55.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:55.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:55.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:55.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:55.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:55.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:55.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:55.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:55.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:55.573588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:55.573588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:55.573588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:55.573588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:55.573588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:55.573588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:55.573588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:55.573588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:55.573588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:55.573588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:55.573588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:42:55.573588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:55.573588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 392/787] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:42:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:55.584764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85cddb-9a52-48e0-bcdd-f243bbd6a11e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:55.615858 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:55.661864 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:55.706315 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d9de1-30b5-4f99-b22e-4e536ebe4bc2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:55.718719 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85cddb-9a52-48e0-bcdd-f243bbd6a11e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:55.744627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64b707-50d3-42a4-955b-37f4343198db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:55.745509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64b707-50d3-42a4-955b-37f4343198db None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:55.745938 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64b707-50d3-42a4-955b-37f4343198db None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:55.746360 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64b707-50d3-42a4-955b-37f4343198db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:55.757864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb64b707-50d3-42a4-955b-37f4343198db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:55.759466 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb64b707-50d3-42a4-955b-37f4343198db None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:55.764777 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30609b-ca21-4d7e-af64-c8c2b7546cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:55.765482 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30609b-ca21-4d7e-af64-c8c2b7546cad None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:55.765966 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30609b-ca21-4d7e-af64-c8c2b7546cad None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:55.766352 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30609b-ca21-4d7e-af64-c8c2b7546cad None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:55.767194 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30609b-ca21-4d7e-af64-c8c2b7546cad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:55.767596 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30609b-ca21-4d7e-af64-c8c2b7546cad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:55.769642 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30609b-ca21-4d7e-af64-c8c2b7546cad None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:55.777108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-bb64b707-50d3-42a4-955b-37f4343198db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:55.777108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:55.777108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:55.777108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:55.777108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:55.777108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:55.777108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:55.777108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:55.777108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:55.777108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:55.777108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:55.777108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:55.777108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:55.777108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:55.777108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:55.777108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:55.778023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:55.778926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:55.778926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:55.778926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:55.781013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 396/789] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:55.782973 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd565b-e389-46e1-a0b6-c03791f5bf15 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:55.783465 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd565b-e389-46e1-a0b6-c03791f5bf15 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:55.783952 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd565b-e389-46e1-a0b6-c03791f5bf15 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:55.784408 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 393/789] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:42:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:55.785779 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30609b-ca21-4d7e-af64-c8c2b7546cad None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:55.786200 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 397/790] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:55.787388 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd565b-e389-46e1-a0b6-c03791f5bf15 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:55.887567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:55.899820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30609b-ca21-4d7e-af64-c8c2b7546cad None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:55.916670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd565b-e389-46e1-a0b6-c03791f5bf15 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:55.917956 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:55.939393 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425ab80-854e-4611-b508-22382cb3439d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:55.940442 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425ab80-854e-4611-b508-22382cb3439d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:55.940995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425ab80-854e-4611-b508-22382cb3439d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:55.941472 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425ab80-854e-4611-b508-22382cb3439d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:55.954528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425ab80-854e-4611-b508-22382cb3439d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:55.955562 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f7a58-34d9-4446-9bd0-75462df62c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:55.956204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f7a58-34d9-4446-9bd0-75462df62c86 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:55.956469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f7a58-34d9-4446-9bd0-75462df62c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:55.956746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f7a58-34d9-4446-9bd0-75462df62c86 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:55.957382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f7a58-34d9-4446-9bd0-75462df62c86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:55.957490 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425ab80-854e-4611-b508-22382cb3439d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:55.957677 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f7a58-34d9-4446-9bd0-75462df62c86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:55.959008 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f7a58-34d9-4446-9bd0-75462df62c86 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:55.971442 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 398/791] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:55.972797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca9bde-5d35-4792-8f91-8bcd261fcd66 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:55.973074 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca9bde-5d35-4792-8f91-8bcd261fcd66 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:55.973343 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca9bde-5d35-4792-8f91-8bcd261fcd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:55.974598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-7425ab80-854e-4611-b508-22382cb3439d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:55.974598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:55.974598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:55.974598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:55.974598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:55.974598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:55.974598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:55.974598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:55.974598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:55.974598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:55.974598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:55.974598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:55.974598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:55.974598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:55.974598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:55.974598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:55.975516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:55.976399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:55.976399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:55.976399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:55.977414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 394/792] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:55.982271 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425ab80-854e-4611-b508-22382cb3439d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:55.998079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-26ca9bde-5d35-4792-8f91-8bcd261fcd66 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:55.999288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-26ca9bde-5d35-4792-8f91-8bcd261fcd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:56.071608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 399/793] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:42:56.072671 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca9bde-5d35-4792-8f91-8bcd261fcd66 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:56.093752 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425ab80-854e-4611-b508-22382cb3439d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:56.131126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd7212-e2d9-4191-8917-fa5f90e9b67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:56.132123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd7212-e2d9-4191-8917-fa5f90e9b67b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:56.132610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd7212-e2d9-4191-8917-fa5f90e9b67b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:56.133107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd7212-e2d9-4191-8917-fa5f90e9b67b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:56.138692 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:56.141808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd7212-e2d9-4191-8917-fa5f90e9b67b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:56.143528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd7212-e2d9-4191-8917-fa5f90e9b67b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:56.155733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-45dd7212-e2d9-4191-8917-fa5f90e9b67b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:42:56.155733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:56.155733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:56.155733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:56.155733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:56.155733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:56.155733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:56.155733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:56.155733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:56.155733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:56.155733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:56.155733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:56.155733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:56.155733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:56.155733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:56.155733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:56.156983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:56.157899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:56.157899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:56.157899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:56.157899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:56.157899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:56.157899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:56.157899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:56.157899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:56.157899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:42:56.157899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:56.160365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 395/794] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 7 07:42:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:56.173686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd7212-e2d9-4191-8917-fa5f90e9b67b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:56.192105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca9bde-5d35-4792-8f91-8bcd261fcd66 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:56.220990 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:56.232004 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66ef16-fa6f-446a-a7b8-befef2cdc91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:56.232699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66ef16-fa6f-446a-a7b8-befef2cdc91e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:56.232978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66ef16-fa6f-446a-a7b8-befef2cdc91e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:56.233247 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66ef16-fa6f-446a-a7b8-befef2cdc91e None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:56.233889 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66ef16-fa6f-446a-a7b8-befef2cdc91e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:56.234189 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66ef16-fa6f-446a-a7b8-befef2cdc91e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:56.235583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66ef16-fa6f-446a-a7b8-befef2cdc91e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:56.263891 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 400/795] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:56.264839 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66ef16-fa6f-446a-a7b8-befef2cdc91e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:56.306106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd7212-e2d9-4191-8917-fa5f90e9b67b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:56.342408 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057ee9c-560f-4a97-9942-1cc6b0e7b9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:56.343347 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057ee9c-560f-4a97-9942-1cc6b0e7b9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:56.343822 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057ee9c-560f-4a97-9942-1cc6b0e7b9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:56.344464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057ee9c-560f-4a97-9942-1cc6b0e7b9a4 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:56.345388 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057ee9c-560f-4a97-9942-1cc6b0e7b9a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:56.345920 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057ee9c-560f-4a97-9942-1cc6b0e7b9a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:56.347529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057ee9c-560f-4a97-9942-1cc6b0e7b9a4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:56.367544 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 396/796] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 7 07:42:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:56.372791 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057ee9c-560f-4a97-9942-1cc6b0e7b9a4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:56.443981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66ef16-fa6f-446a-a7b8-befef2cdc91e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:56.477064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:56.492007 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a70a08-2523-414a-a2f7-66c0a9abfebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:56.492766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a70a08-2523-414a-a2f7-66c0a9abfebb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:56.493051 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a70a08-2523-414a-a2f7-66c0a9abfebb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:56.493330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a70a08-2523-414a-a2f7-66c0a9abfebb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:56.507056 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a70a08-2523-414a-a2f7-66c0a9abfebb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:56.508565 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a70a08-2523-414a-a2f7-66c0a9abfebb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:56.522120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-a2a70a08-2523-414a-a2f7-66c0a9abfebb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 07:42:56.522120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:56.522120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:56.522120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:56.522120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:56.522120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:56.522120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:56.522120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:56.522120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:56.522120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:56.522120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:56.522120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:56.522120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:56.522120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:56.522120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:56.522120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:56.523158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:56.524159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:56.524159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:56.524159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:56.524159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:56.524159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:56.524159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:56.524159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:56.524159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:56.524159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 07:42:56.524159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:56.524159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 401/797] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:42:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:56.524159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a70a08-2523-414a-a2f7-66c0a9abfebb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:56.532165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:56.554818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057ee9c-560f-4a97-9942-1cc6b0e7b9a4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:56.603227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e976a1-7ac1-42ff-b3e6-af65bbc4d960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:56.604112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e976a1-7ac1-42ff-b3e6-af65bbc4d960 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:56.604569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e976a1-7ac1-42ff-b3e6-af65bbc4d960 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:56.604993 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e976a1-7ac1-42ff-b3e6-af65bbc4d960 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:56.633219 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a70a08-2523-414a-a2f7-66c0a9abfebb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:56.669735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e5e56-4000-4b68-941c-35dad0f2fa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:56.670377 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e5e56-4000-4b68-941c-35dad0f2fa38 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:56.670654 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e5e56-4000-4b68-941c-35dad0f2fa38 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:56.670922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e5e56-4000-4b68-941c-35dad0f2fa38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:56.682100 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e976a1-7ac1-42ff-b3e6-af65bbc4d960 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:56.682456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e5e56-4000-4b68-941c-35dad0f2fa38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:42:56.683563 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e976a1-7ac1-42ff-b3e6-af65bbc4d960 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:56.688346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-b99e5e56-4000-4b68-941c-35dad0f2fa38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:56.688346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:56.688346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:56.688346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:56.688346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:56.688346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:56.688346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:56.688346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:56.688346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:56.688346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:56.688346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:56.688346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:56.688346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:42:56.688346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:42:56.688346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:42:56.688346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:42:56.689439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:42:56.689439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:42:56.689439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:42:56.689439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:42:56.689439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:56.689439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:56.689439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:56.689439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:56.689439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:56.689439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:56.689439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:56.689439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:56.689439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:56.689439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:56.689439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:56.689439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:56.690285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:56.690285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:56.690285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:56.690285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:56.690285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:42:56.690285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:42:56.690285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:42:56.690285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:42:56.690285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:42:56.690285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:42:56.690285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:42:56.690285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:56.690285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 402/798] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:42:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:56.691024 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e5e56-4000-4b68-941c-35dad0f2fa38 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:56.739905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:56.780006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:56.801208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 397/799] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:42:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:42:56.801208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89f10a-dbb1-41a7-970a-1aeb154677ea None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:56.801208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89f10a-dbb1-41a7-970a-1aeb154677ea None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:56.801208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89f10a-dbb1-41a7-970a-1aeb154677ea None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:56.801208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 398/800] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:56.801208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89f10a-dbb1-41a7-970a-1aeb154677ea None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:56.811661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e5e56-4000-4b68-941c-35dad0f2fa38 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:56.849001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e40eb8-0304-4a3d-9e1a-25717dae8798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:56.849610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e40eb8-0304-4a3d-9e1a-25717dae8798 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:56.849885 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e40eb8-0304-4a3d-9e1a-25717dae8798 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:56.850164 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e40eb8-0304-4a3d-9e1a-25717dae8798 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:56.850887 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e40eb8-0304-4a3d-9e1a-25717dae8798 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:56.851193 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e40eb8-0304-4a3d-9e1a-25717dae8798 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:56.852647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e40eb8-0304-4a3d-9e1a-25717dae8798 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:56.880408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 403/801] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Sep 7 07:42:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:56.880408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e40eb8-0304-4a3d-9e1a-25717dae8798 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:56.917402 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89f10a-dbb1-41a7-970a-1aeb154677ea None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:56.955307 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6854a-ebb9-4dfe-8242-a808bc955b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:56.955907 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6854a-ebb9-4dfe-8242-a808bc955b52 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:56.956209 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6854a-ebb9-4dfe-8242-a808bc955b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:56.956477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6854a-ebb9-4dfe-8242-a808bc955b52 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:56.957123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b6854a-ebb9-4dfe-8242-a808bc955b52 None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:56.957419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b6854a-ebb9-4dfe-8242-a808bc955b52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:56.958792 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b6854a-ebb9-4dfe-8242-a808bc955b52 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:56.968583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 399/802] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:42:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:56.970103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102c086-8c3e-4d22-8f61-51df60427667 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:56.970384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102c086-8c3e-4d22-8f61-51df60427667 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:56.970656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102c086-8c3e-4d22-8f61-51df60427667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:56.990164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3102c086-8c3e-4d22-8f61-51df60427667 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:56.991453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3102c086-8c3e-4d22-8f61-51df60427667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:56.991696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e40eb8-0304-4a3d-9e1a-25717dae8798 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:56.993736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:57.034079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4bb2c-4cae-48d7-9f9d-fb03834462f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:57.034722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4bb2c-4cae-48d7-9f9d-fb03834462f4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:57.034994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4bb2c-4cae-48d7-9f9d-fb03834462f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:57.035260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4bb2c-4cae-48d7-9f9d-fb03834462f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:57.045210 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:57.046780 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4bb2c-4cae-48d7-9f9d-fb03834462f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:57.049893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4bb2c-4cae-48d7-9f9d-fb03834462f4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:57.065206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-45f4bb2c-4cae-48d7-9f9d-fb03834462f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:57.065206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:57.065206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:57.065206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:57.065206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:57.065206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:57.065206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:57.065206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:57.065206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:57.065206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:57.065206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:57.065206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:57.065206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:57.065206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:57.065206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:57.065206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:57.066541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:57.067470 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:57.067470 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:57.067470 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:57.071518 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 404/803] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:57.073009 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4bb2c-4cae-48d7-9f9d-fb03834462f4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:57.092292 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 400/804] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:42:57.092292 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102c086-8c3e-4d22-8f61-51df60427667 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:57.195595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4bb2c-4cae-48d7-9f9d-fb03834462f4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:57.217593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102c086-8c3e-4d22-8f61-51df60427667 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:57.232565 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0a6dc-f89e-4ee3-92e6-c0c0e50b31c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:57.233178 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0a6dc-f89e-4ee3-92e6-c0c0e50b31c6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:57.233456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0a6dc-f89e-4ee3-92e6-c0c0e50b31c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:57.233727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0a6dc-f89e-4ee3-92e6-c0c0e50b31c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:57.246230 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0a6dc-f89e-4ee3-92e6-c0c0e50b31c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:57.248146 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0a6dc-f89e-4ee3-92e6-c0c0e50b31c6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:57.259037 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718784f1-7815-4a92-8425-a4e6fc60ac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:57.259989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718784f1-7815-4a92-8425-a4e6fc60ac50 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:57.260297 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718784f1-7815-4a92-8425-a4e6fc60ac50 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:57.260571 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718784f1-7815-4a92-8425-a4e6fc60ac50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:57.263670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-e3d0a6dc-f89e-4ee3-92e6-c0c0e50b31c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:57.263670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:57.263670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:57.263670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:57.263670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:57.263670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:57.263670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:57.263670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:57.263670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:57.263670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:57.263670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:57.263670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:57.263670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:57.263670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:57.263670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:57.263670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:57.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:57.266616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:57.266616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:57.266616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:57.266616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 405/805] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:57.266616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0a6dc-f89e-4ee3-92e6-c0c0e50b31c6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:57.277548 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:57.280981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718784f1-7815-4a92-8425-a4e6fc60ac50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:57.284692 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718784f1-7815-4a92-8425-a4e6fc60ac50 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:57.304808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-718784f1-7815-4a92-8425-a4e6fc60ac50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:57.304808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:57.304808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:57.304808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:57.304808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:57.304808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:57.304808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:57.304808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:57.304808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:57.304808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:57.304808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:57.304808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:57.304808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:57.304808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:57.304808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:57.304808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:57.305798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:57.306798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:57.306798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:57.306798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:57.306798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 401/806] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:57.307811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07539a67-fb10-4ea3-920d-38fff2b7e2bf None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:57.308355 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07539a67-fb10-4ea3-920d-38fff2b7e2bf None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:57.308514 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07539a67-fb10-4ea3-920d-38fff2b7e2bf None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:57.310001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 402/807] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:42:57.310632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07539a67-fb10-4ea3-920d-38fff2b7e2bf None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:57.344818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:57.401544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0a6dc-f89e-4ee3-92e6-c0c0e50b31c6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:57.430295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07539a67-fb10-4ea3-920d-38fff2b7e2bf None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:57.440973 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8569f9d-1efb-41e8-86c3-58d75aa73c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:57.441943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8569f9d-1efb-41e8-86c3-58d75aa73c21 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:57.442213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8569f9d-1efb-41e8-86c3-58d75aa73c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:57.442477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8569f9d-1efb-41e8-86c3-58d75aa73c21 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:57.443420 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8569f9d-1efb-41e8-86c3-58d75aa73c21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:57.443717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8569f9d-1efb-41e8-86c3-58d75aa73c21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:57.445368 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8569f9d-1efb-41e8-86c3-58d75aa73c21 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:57.458250 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 406/808] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:57.459360 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8569f9d-1efb-41e8-86c3-58d75aa73c21 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:57.471644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbf083-cfc2-43cd-ad28-db728f515295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:57.472710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbf083-cfc2-43cd-ad28-db728f515295 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:57.473087 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbf083-cfc2-43cd-ad28-db728f515295 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:57.473361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbf083-cfc2-43cd-ad28-db728f515295 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:57.473998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbf083-cfc2-43cd-ad28-db728f515295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:57.474300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbf083-cfc2-43cd-ad28-db728f515295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:57.475676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbf083-cfc2-43cd-ad28-db728f515295 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:57.491689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 403/809] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:57.493028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741a301-690c-4846-a2d5-b412b81e59b1 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:57.493341 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741a301-690c-4846-a2d5-b412b81e59b1 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:57.493610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741a301-690c-4846-a2d5-b412b81e59b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:57.512687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e741a301-690c-4846-a2d5-b412b81e59b1 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:42:57.513817 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e741a301-690c-4846-a2d5-b412b81e59b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:42:57.531017 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:57.568012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8569f9d-1efb-41e8-86c3-58d75aa73c21 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:57.584658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 404/810] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:42:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:42:57.585967 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741a301-690c-4846-a2d5-b412b81e59b1 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:57.611117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d60556-7412-480a-a83b-638de73cf7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:57.611736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d60556-7412-480a-a83b-638de73cf7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:57.612013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d60556-7412-480a-a83b-638de73cf7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:57.612327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d60556-7412-480a-a83b-638de73cf7b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:57.612989 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d60556-7412-480a-a83b-638de73cf7b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:57.613294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d60556-7412-480a-a83b-638de73cf7b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:57.614655 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d60556-7412-480a-a83b-638de73cf7b4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:57.626680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:57.627212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 407/811] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:57.628283 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d60556-7412-480a-a83b-638de73cf7b4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:57.733164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741a301-690c-4846-a2d5-b412b81e59b1 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:57.739290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d60556-7412-480a-a83b-638de73cf7b4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:57.774441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2026c7a-e9f5-4eff-b968-90a34d1caf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:57.774989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2026c7a-e9f5-4eff-b968-90a34d1caf81 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:57.775173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:57.775285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2026c7a-e9f5-4eff-b968-90a34d1caf81 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:57.775510 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2026c7a-e9f5-4eff-b968-90a34d1caf81 None admin] PATH_INFO: `/v3/services/network` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:57.776102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2026c7a-e9f5-4eff-b968-90a34d1caf81 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:57.777557 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2026c7a-e9f5-4eff-b968-90a34d1caf81 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:57.786445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e32e2a-7ed8-4a98-9d0f-5dd51c8ba531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:57.787363 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e32e2a-7ed8-4a98-9d0f-5dd51c8ba531 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-d2026c7a-e9f5-4eff-b968-90a34d1caf81 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:57.787474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:57.788461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:57.788461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:57.788461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:57.788461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:57.788461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:57.788461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:57.788461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:42:57.788461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:42:57.788461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:42:57.788461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:42:57.788461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:57.788461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:57.788461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:57.788461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:57.788461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:57.788461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:57.789300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:57.789300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:57.789300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:57.789300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:57.789300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:57.789300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:57.789300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:57.789300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:57.789300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:57.789300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:57.789300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:42:57.789300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:42:57.789300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 07:42:57.789300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:57.789300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 405/812] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:57 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:57.790132 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2026c7a-e9f5-4eff-b968-90a34d1caf81 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:57.790772 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e32e2a-7ed8-4a98-9d0f-5dd51c8ba531 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:57.791057 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e32e2a-7ed8-4a98-9d0f-5dd51c8ba531 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:57.800103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e32e2a-7ed8-4a98-9d0f-5dd51c8ba531 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:57.801654 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e32e2a-7ed8-4a98-9d0f-5dd51c8ba531 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:57.814641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-42e32e2a-7ed8-4a98-9d0f-5dd51c8ba531 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:57.814641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:57.814641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:57.814641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:57.814641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:57.814641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:57.814641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:57.814641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:57.814641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:57.814641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:57.814641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:57.814641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:57.814641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:57.814641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:57.814641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:57.815611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:57.816502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:57.816502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:57.816502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:57.816502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:57.816502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:57.816502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:57.816502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:57.816502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:57.816502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 408/813] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:57.816972 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e32e2a-7ed8-4a98-9d0f-5dd51c8ba531 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:57.902214 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:57.918931 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2026c7a-e9f5-4eff-b968-90a34d1caf81 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:57.924315 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e32e2a-7ed8-4a98-9d0f-5dd51c8ba531 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:57.952888 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ba39b-8b9c-4374-88c9-cc6a9a5a4edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:57.953657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ba39b-8b9c-4374-88c9-cc6a9a5a4edd None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:57.953934 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ba39b-8b9c-4374-88c9-cc6a9a5a4edd None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:57.954203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ba39b-8b9c-4374-88c9-cc6a9a5a4edd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:57.958585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e23c40-4bf9-4078-945e-e5710624f03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:57.959209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e23c40-4bf9-4078-945e-e5710624f03a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:57.959484 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e23c40-4bf9-4078-945e-e5710624f03a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:57.959755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e23c40-4bf9-4078-945e-e5710624f03a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:57.961066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ba39b-8b9c-4374-88c9-cc6a9a5a4edd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:57.962514 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ba39b-8b9c-4374-88c9-cc6a9a5a4edd None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:57.967298 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e23c40-4bf9-4078-945e-e5710624f03a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:57.968745 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e23c40-4bf9-4078-945e-e5710624f03a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:57.974361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-027ba39b-8b9c-4374-88c9-cc6a9a5a4edd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:57.974361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:57.974361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:57.974361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:57.974361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:57.974361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:57.974361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:57.974361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:57.974361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:57.974361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:57.974361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:57.974361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:57.974361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:57.974361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:57.974361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:57.975278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:57.976199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:57.976199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:57.976199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:57.976199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:57.976199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:57.976199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:57.976199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:57.976199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:57.976199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 406/814] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:57.976685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ba39b-8b9c-4374-88c9-cc6a9a5a4edd None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:57.980708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d3e23c40-4bf9-4078-945e-e5710624f03a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 07:42:57.980708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:57.980708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:57.980708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:57.980708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:57.980708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:57.980708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:57.980708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:57.980708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:57.980708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:57.980708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:57.980708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:57.980708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:42:57.980708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:42:57.980708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:42:57.980708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:57.981727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:57.982871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:57.982871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:57.982871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:42:57.982871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:42:57.982871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:42:57.982871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:42:57.982871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:42:57.982871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:42:57.982871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 07:42:57.982871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:57.982871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 409/815] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 7 07:42:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:57.982871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e23c40-4bf9-4078-945e-e5710624f03a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:58.027652 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:58.080498 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ba39b-8b9c-4374-88c9-cc6a9a5a4edd None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:58.080943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e23c40-4bf9-4078-945e-e5710624f03a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:58.115473 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd2d1f-bb59-4eb5-9c69-dfdb11268024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:58.115718 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b104843-d977-4365-8450-5343ad62c778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:58.116068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd2d1f-bb59-4eb5-9c69-dfdb11268024 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:58.116230 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b104843-d977-4365-8450-5343ad62c778 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:58.116370 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd2d1f-bb59-4eb5-9c69-dfdb11268024 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:58.116463 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b104843-d977-4365-8450-5343ad62c778 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:58.116634 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd2d1f-bb59-4eb5-9c69-dfdb11268024 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:58.116745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b104843-d977-4365-8450-5343ad62c778 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:58.117297 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b104843-d977-4365-8450-5343ad62c778 None admin] RBAC: Adding query filter params (name=network) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:58.117396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd2d1f-bb59-4eb5-9c69-dfdb11268024 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:58.117595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b104843-d977-4365-8450-5343ad62c778 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:58.117686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd2d1f-bb59-4eb5-9c69-dfdb11268024 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:58.118951 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b104843-d977-4365-8450-5343ad62c778 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:58.119049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd2d1f-bb59-4eb5-9c69-dfdb11268024 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:58.123314 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 407/816] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 7 07:42:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:58.124071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b104843-d977-4365-8450-5343ad62c778 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:58.131087 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 410/817] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:58.131885 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd2d1f-bb59-4eb5-9c69-dfdb11268024 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:58.136294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:58.227403 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b104843-d977-4365-8450-5343ad62c778 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:58.228577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd2d1f-bb59-4eb5-9c69-dfdb11268024 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:58.235910 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:58.261380 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5e143-a2e0-47e3-a5b8-c8b6e91c5134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:58.262013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5e143-a2e0-47e3-a5b8-c8b6e91c5134 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:58.262291 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5e143-a2e0-47e3-a5b8-c8b6e91c5134 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:58.262561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5e143-a2e0-47e3-a5b8-c8b6e91c5134 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:58.263064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79d2ab-8314-4719-8406-2c4fe70c27e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:58.263173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5e143-a2e0-47e3-a5b8-c8b6e91c5134 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:58.263654 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79d2ab-8314-4719-8406-2c4fe70c27e1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:58.263925 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79d2ab-8314-4719-8406-2c4fe70c27e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:58.264229 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79d2ab-8314-4719-8406-2c4fe70c27e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:58.264733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5e143-a2e0-47e3-a5b8-c8b6e91c5134 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:58.264910 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a79d2ab-8314-4719-8406-2c4fe70c27e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:58.265209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a79d2ab-8314-4719-8406-2c4fe70c27e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:58.267098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a79d2ab-8314-4719-8406-2c4fe70c27e1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-87c5e143-a2e0-47e3-a5b8-c8b6e91c5134 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:58.274012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:58.274970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:58.274970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:58.274970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:58.274970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:58.274970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:58.274970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:58.274970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:42:58.274970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:42:58.274970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:42:58.274970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:42:58.274970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:58.274970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:58.274970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:58.274970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:58.274970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:58.274970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:58.275804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:58.275804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:58.275804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:58.275804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:58.275804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:58.275804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:58.275804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:58.275804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:58.275804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:58.275804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:58.275804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:42:58.275804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:42:58.275804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 07:42:58.275804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:42:58.275804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 408/818] 10.222.0.26 () {56 vars in 1129 bytes} [Wed Sep 7 07:42:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:58.277239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5e143-a2e0-47e3-a5b8-c8b6e91c5134 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:58.278779 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 411/819] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:58.279546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79d2ab-8314-4719-8406-2c4fe70c27e1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:58.351364 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:58.381673 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5e143-a2e0-47e3-a5b8-c8b6e91c5134 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:58.382770 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79d2ab-8314-4719-8406-2c4fe70c27e1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:58.423675 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc0ed4-5ddb-4fbf-ba96-be4a70fe72ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:58.424325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc0ed4-5ddb-4fbf-ba96-be4a70fe72ef None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:58.424601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc0ed4-5ddb-4fbf-ba96-be4a70fe72ef None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:58.424867 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc0ed4-5ddb-4fbf-ba96-be4a70fe72ef None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:58.425497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc0ed4-5ddb-4fbf-ba96-be4a70fe72ef None admin] RBAC: Adding query filter params (type=network) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:58.425799 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc0ed4-5ddb-4fbf-ba96-be4a70fe72ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:58.427178 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc0ed4-5ddb-4fbf-ba96-be4a70fe72ef None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:58.432719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 412/820] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 7 07:42:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:58.433582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc0ed4-5ddb-4fbf-ba96-be4a70fe72ef None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:58.466218 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:58.530759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf9dc04-5df7-4564-abf5-686b54585efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:58.531273 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf9dc04-5df7-4564-abf5-686b54585efe None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:58.532224 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf9dc04-5df7-4564-abf5-686b54585efe None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:58.532499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf9dc04-5df7-4564-abf5-686b54585efe None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:58.533209 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf9dc04-5df7-4564-abf5-686b54585efe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:58.533512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf9dc04-5df7-4564-abf5-686b54585efe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:58.534929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf9dc04-5df7-4564-abf5-686b54585efe None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:58.538261 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc0ed4-5ddb-4fbf-ba96-be4a70fe72ef None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:58.552342 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 409/821] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Sep 7 07:42:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:58.552971 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf9dc04-5df7-4564-abf5-686b54585efe None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:58.575700 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec14bd-4847-4916-9188-ef9a6efd53d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:58.576339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec14bd-4847-4916-9188-ef9a6efd53d6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:58.576614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec14bd-4847-4916-9188-ef9a6efd53d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:58.576882 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec14bd-4847-4916-9188-ef9a6efd53d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:58.577585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec14bd-4847-4916-9188-ef9a6efd53d6 None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=81d5689e9a344bafb74ca423ffd2b42a) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:58.577887 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec14bd-4847-4916-9188-ef9a6efd53d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:58.579288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec14bd-4847-4916-9188-ef9a6efd53d6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:58.584940 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 413/822] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:42:58 2022] GET /identity/v3/role_assignments?user.id=81d5689e9a344bafb74ca423ffd2b42a&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:58.585787 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec14bd-4847-4916-9188-ef9a6efd53d6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:58.661245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf9dc04-5df7-4564-abf5-686b54585efe None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:58.694865 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:58.701652 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc255d1b-47e4-42fc-9fe4-87369d673d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:58.702271 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc255d1b-47e4-42fc-9fe4-87369d673d33 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:58.702542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc255d1b-47e4-42fc-9fe4-87369d673d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:58.702927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc255d1b-47e4-42fc-9fe4-87369d673d33 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:58.703507 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc255d1b-47e4-42fc-9fe4-87369d673d33 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:58.703802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc255d1b-47e4-42fc-9fe4-87369d673d33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:58.705216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc255d1b-47e4-42fc-9fe4-87369d673d33 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:58.708183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:58.710473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 410/823] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Sep 7 07:42:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:58.716307 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc255d1b-47e4-42fc-9fe4-87369d673d33 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:58.719812 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec14bd-4847-4916-9188-ef9a6efd53d6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:58.763112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c280dd-1a6a-422f-b773-8325263b33aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:58.763772 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c280dd-1a6a-422f-b773-8325263b33aa None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:58.764058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c280dd-1a6a-422f-b773-8325263b33aa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:58.764498 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c280dd-1a6a-422f-b773-8325263b33aa None admin] PATH_INFO: `/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/3ff3bf8da732489b8b3a8fbe5aa22f21/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:58.766793 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c280dd-1a6a-422f-b773-8325263b33aa None admin] RBAC: Authorizing `identity:create_grant(project_id=9a949b959e584474aafb4a05ebbabcc4, user_id=3ff3bf8da732489b8b3a8fbe5aa22f21, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:58.768147 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c280dd-1a6a-422f-b773-8325263b33aa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:58.783556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 414/824] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:42:58 2022] PUT /identity/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/3ff3bf8da732489b8b3a8fbe5aa22f21/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:42:58.788135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c280dd-1a6a-422f-b773-8325263b33aa None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:58.849808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc255d1b-47e4-42fc-9fe4-87369d673d33 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:58.897787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc077e-e441-4594-ad8d-229904a8ab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:58.898416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc077e-e441-4594-ad8d-229904a8ab87 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:58.898694 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc077e-e441-4594-ad8d-229904a8ab87 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:58.898961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc077e-e441-4594-ad8d-229904a8ab87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:58.899647 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc077e-e441-4594-ad8d-229904a8ab87 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:58.901392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc077e-e441-4594-ad8d-229904a8ab87 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:58.919692 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c280dd-1a6a-422f-b773-8325263b33aa None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:58.936727 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 411/825] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:42:58 2022] POST /identity/v3/endpoints => generated 337 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:42:58.938091 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc077e-e441-4594-ad8d-229904a8ab87 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:58.987333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfe7e8-43ab-47f1-a8fa-b9f77e0da982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:58.988365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfe7e8-43ab-47f1-a8fa-b9f77e0da982 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:58.988785 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfe7e8-43ab-47f1-a8fa-b9f77e0da982 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:58.989205 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfe7e8-43ab-47f1-a8fa-b9f77e0da982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:59.005140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfe7e8-43ab-47f1-a8fa-b9f77e0da982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:59.006661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfe7e8-43ab-47f1-a8fa-b9f77e0da982 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:59.023022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-cfcfe7e8-43ab-47f1-a8fa-b9f77e0da982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:59.023022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:59.023022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:59.023022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:59.023022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:59.023022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:59.023022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:59.023022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:59.023022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:59.023022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:59.023022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:59.023022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:59.023022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:42:59.023022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:42:59.023022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:42:59.023022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:59.024033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:42:59.024981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:42:59.024981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:42:59.024981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:59.024981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 415/826] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:59.026520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:59.032149 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfe7e8-43ab-47f1-a8fa-b9f77e0da982 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:59.036543 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:59.137294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc077e-e441-4594-ad8d-229904a8ab87 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:59.148697 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfe7e8-43ab-47f1-a8fa-b9f77e0da982 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:59.174753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982d69e-8978-43ba-8681-8afa6b74c301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:59.175362 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982d69e-8978-43ba-8681-8afa6b74c301 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:59.175636 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982d69e-8978-43ba-8681-8afa6b74c301 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:59.175902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982d69e-8978-43ba-8681-8afa6b74c301 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:59.176554 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982d69e-8978-43ba-8681-8afa6b74c301 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:59.176849 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982d69e-8978-43ba-8681-8afa6b74c301 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:59.178212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982d69e-8978-43ba-8681-8afa6b74c301 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:59.188157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 412/827] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Sep 7 07:42:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:59.189248 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2405fd8-c9ff-47d1-9abd-56c7b2b390b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:59.189412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982d69e-8978-43ba-8681-8afa6b74c301 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:59.190052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2405fd8-c9ff-47d1-9abd-56c7b2b390b7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:59.190433 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2405fd8-c9ff-47d1-9abd-56c7b2b390b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:59.190779 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2405fd8-c9ff-47d1-9abd-56c7b2b390b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:59.191513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2405fd8-c9ff-47d1-9abd-56c7b2b390b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:59.192014 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2405fd8-c9ff-47d1-9abd-56c7b2b390b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:59.193577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2405fd8-c9ff-47d1-9abd-56c7b2b390b7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:59.208324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 416/828] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:59.220518 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2405fd8-c9ff-47d1-9abd-56c7b2b390b7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:59.302397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:59.318002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982d69e-8978-43ba-8681-8afa6b74c301 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:59.337552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:59.361957 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2405fd8-c9ff-47d1-9abd-56c7b2b390b7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:59.368916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c8ec8-346b-4373-9001-0a85fe26a6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:59.369660 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c8ec8-346b-4373-9001-0a85fe26a6d1 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:59.370034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c8ec8-346b-4373-9001-0a85fe26a6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:59.370891 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c8ec8-346b-4373-9001-0a85fe26a6d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:59.371671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c8ec8-346b-4373-9001-0a85fe26a6d1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:59.373557 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c8ec8-346b-4373-9001-0a85fe26a6d1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:59.411985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93456b13-f96d-40be-96b7-b151a6f375a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:59.413145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93456b13-f96d-40be-96b7-b151a6f375a9 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:59.413564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93456b13-f96d-40be-96b7-b151a6f375a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:59.413986 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93456b13-f96d-40be-96b7-b151a6f375a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:59.420112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 413/829] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:42:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:42:59.427541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93456b13-f96d-40be-96b7-b151a6f375a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:59.429238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93456b13-f96d-40be-96b7-b151a6f375a9 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:59.443701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-93456b13-f96d-40be-96b7-b151a6f375a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:59.443701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:42:59.443701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:42:59.443701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:42:59.443701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:42:59.443701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:42:59.443701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:42:59.443701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:42:59.443701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:42:59.443701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:42:59.443701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:42:59.443701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:42:59.443701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:42:59.443701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:42:59.443701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:42:59.445148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:42:59.446220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:42:59.446220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:42:59.446220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:42:59.446220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:42:59.446220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:42:59.446220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:42:59.446220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:42:59.446220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:42:59.448347 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 417/830] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:42:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:42:59.453743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c8ec8-346b-4373-9001-0a85fe26a6d1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:59.579286 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c8ec8-346b-4373-9001-0a85fe26a6d1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:59.617532 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064c2fc-abd4-4f9f-9693-71d24011ce5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:59.618196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064c2fc-abd4-4f9f-9693-71d24011ce5c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:59.618477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064c2fc-abd4-4f9f-9693-71d24011ce5c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:59.618747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064c2fc-abd4-4f9f-9693-71d24011ce5c None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:59.619443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064c2fc-abd4-4f9f-9693-71d24011ce5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:42:59.619747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064c2fc-abd4-4f9f-9693-71d24011ce5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:59.621234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064c2fc-abd4-4f9f-9693-71d24011ce5c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:59.635821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 414/831] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:42:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:42:59.642063 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93456b13-f96d-40be-96b7-b151a6f375a9 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:42:59.750335 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93456b13-f96d-40be-96b7-b151a6f375a9 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:42:59.783724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:42:59.786838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453802f-330b-4ff5-a8b0-be3a090d50dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:42:59.787569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453802f-330b-4ff5-a8b0-be3a090d50dc None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:42:59.787914 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453802f-330b-4ff5-a8b0-be3a090d50dc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:42:59.788509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453802f-330b-4ff5-a8b0-be3a090d50dc None admin] PATH_INFO: `/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/617f6479b90945d3b5a63325839afb15/roles/e7960e6d957f4a1fa6547565db026f24` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:42:59.804816 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453802f-330b-4ff5-a8b0-be3a090d50dc None admin] RBAC: Authorizing `identity:create_grant(project_id=9a949b959e584474aafb4a05ebbabcc4, user_id=617f6479b90945d3b5a63325839afb15, role_id=e7960e6d957f4a1fa6547565db026f24)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:42:59.806321 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453802f-330b-4ff5-a8b0-be3a090d50dc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:42:59.821595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 418/832] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:42:59 2022] PUT /identity/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/617f6479b90945d3b5a63325839afb15/roles/e7960e6d957f4a1fa6547565db026f24 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:43:00.998468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bdb3c-7c4b-4b64-ae7d-699ac6a6042c None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:00.999015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bdb3c-7c4b-4b64-ae7d-699ac6a6042c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:00.999015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bdb3c-7c4b-4b64-ae7d-699ac6a6042c None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:01.001310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 415/833] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:01.006861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd78328-899f-4eb3-8b0b-46d29fc493e0 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:01.007247 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd78328-899f-4eb3-8b0b-46d29fc493e0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:01.007428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd78328-899f-4eb3-8b0b-46d29fc493e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:01.025552 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cfd78328-899f-4eb3-8b0b-46d29fc493e0 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:01.026725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cfd78328-899f-4eb3-8b0b-46d29fc493e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:01.105149 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 419/834] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:01.187177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdd239-1c85-48ae-a11d-bc115f81336f None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:01.187531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdd239-1c85-48ae-a11d-bc115f81336f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:01.187834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdd239-1c85-48ae-a11d-bc115f81336f None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:01.189742 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 416/835] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:01.195276 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed3392-b96a-4f16-a74c-037cacd423df None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:01.195565 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed3392-b96a-4f16-a74c-037cacd423df None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:01.195836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed3392-b96a-4f16-a74c-037cacd423df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:01.215800 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-deed3392-b96a-4f16-a74c-037cacd423df None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:01.217011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-deed3392-b96a-4f16-a74c-037cacd423df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:01.285623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 420/836] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:01.373501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdd239-1c85-48ae-a11d-bc115f81336f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:01.408231 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:01.495858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdd239-1c85-48ae-a11d-bc115f81336f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:01.532457 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a8172-c9fc-43b6-b269-6f5a8baefe1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:01.533081 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a8172-c9fc-43b6-b269-6f5a8baefe1e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:01.533356 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a8172-c9fc-43b6-b269-6f5a8baefe1e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:01.533623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a8172-c9fc-43b6-b269-6f5a8baefe1e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:01.534199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a8172-c9fc-43b6-b269-6f5a8baefe1e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:01.535558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a8172-c9fc-43b6-b269-6f5a8baefe1e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:01.544686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-8d5a8172-c9fc-43b6-b269-6f5a8baefe1e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 07:43:01.544686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:01.544686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:01.544686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:01.544686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:01.544686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:01.544686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:01.544686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:01.544686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 07:43:01.544686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:01.544686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:01.544686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:01.544686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:01.544686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:01.544686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:01.544686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:01.545612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:01.546488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:01.546488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:01.546488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:01.546488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:01.546488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:01.546488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:01.546488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:01.546488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:01.546488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:01.546488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:01.546488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:01.546488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:01.546488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 07:43:01.546488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:01.546488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 417/837] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Sep 7 07:43:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:01.552410 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed3392-b96a-4f16-a74c-037cacd423df None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:01.571358 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a8172-c9fc-43b6-b269-6f5a8baefe1e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:01.588246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:01.672035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed3392-b96a-4f16-a74c-037cacd423df None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:01.708526 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:01.708725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada48095-a090-4f41-ad5d-2f724d1a1310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:01.709686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada48095-a090-4f41-ad5d-2f724d1a1310 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:01.710052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada48095-a090-4f41-ad5d-2f724d1a1310 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:01.710451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada48095-a090-4f41-ad5d-2f724d1a1310 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:01.711076 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada48095-a090-4f41-ad5d-2f724d1a1310 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:01.715604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada48095-a090-4f41-ad5d-2f724d1a1310 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:01.717437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada48095-a090-4f41-ad5d-2f724d1a1310 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:01.723047 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a8172-c9fc-43b6-b269-6f5a8baefe1e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:01.728182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 421/838] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 7 07:43:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:01.729895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada48095-a090-4f41-ad5d-2f724d1a1310 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:01.765219 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73765a06-172f-4365-aea6-c2f07877c5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:01.765965 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73765a06-172f-4365-aea6-c2f07877c5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:01.766443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73765a06-172f-4365-aea6-c2f07877c5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:01.766897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73765a06-172f-4365-aea6-c2f07877c5c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:01.777325 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73765a06-172f-4365-aea6-c2f07877c5c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:43:01.783542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-73765a06-172f-4365-aea6-c2f07877c5c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:43:01.783542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:01.783542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:01.783542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:01.783542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:01.783542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:01.783542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:01.783542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:01.783542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:01.783542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:01.783542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:01.783542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:01.783542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:43:01.783542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:43:01.783542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:43:01.783542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:43:01.784535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:43:01.784535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:43:01.784535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:43:01.784535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:43:01.784535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:01.784535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:01.784535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:01.784535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:01.784535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:01.784535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:01.784535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:01.784535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:01.784535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:01.784535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:01.784535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:01.784535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:01.785365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:01.785365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:01.785365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:01.785365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:01.785365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:43:01.785365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:43:01.785365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:43:01.785365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:43:01.785365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:43:01.785365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:43:01.785365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:43:01.785365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:01.789627 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 418/839] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:43:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:01.790676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1af93c-7275-4905-b4c3-2dbd8d49ccff None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:01.790962 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1af93c-7275-4905-b4c3-2dbd8d49ccff None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:01.791246 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1af93c-7275-4905-b4c3-2dbd8d49ccff None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:01.792648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 419/840] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:01.793210 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1af93c-7275-4905-b4c3-2dbd8d49ccff None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:01.842869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:01.849915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada48095-a090-4f41-ad5d-2f724d1a1310 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:01.887194 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad61ba-a624-4203-829b-b73031f12918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:01.887824 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad61ba-a624-4203-829b-b73031f12918 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:01.888133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad61ba-a624-4203-829b-b73031f12918 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:01.888409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad61ba-a624-4203-829b-b73031f12918 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:01.889149 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad61ba-a624-4203-829b-b73031f12918 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:01.889474 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad61ba-a624-4203-829b-b73031f12918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:01.891162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad61ba-a624-4203-829b-b73031f12918 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:01.896637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 422/841] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 7 07:43:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:01.897780 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00b269-11a0-4005-9a57-732d1071b859 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:01.898064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00b269-11a0-4005-9a57-732d1071b859 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:01.898340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00b269-11a0-4005-9a57-732d1071b859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:01.912413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1af93c-7275-4905-b4c3-2dbd8d49ccff None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:01.927047 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-bb00b269-11a0-4005-9a57-732d1071b859 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:01.928378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bb00b269-11a0-4005-9a57-732d1071b859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:01.963478 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0818f8-cb4d-420f-995e-a97b1b8bc76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:01.964499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0818f8-cb4d-420f-995e-a97b1b8bc76c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:01.965036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0818f8-cb4d-420f-995e-a97b1b8bc76c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:01.965561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0818f8-cb4d-420f-995e-a97b1b8bc76c None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:01.966675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0818f8-cb4d-420f-995e-a97b1b8bc76c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:01.967173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0818f8-cb4d-420f-995e-a97b1b8bc76c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:01.969613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0818f8-cb4d-420f-995e-a97b1b8bc76c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:01.981380 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 420/842] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:43:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:01.988256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0818f8-cb4d-420f-995e-a97b1b8bc76c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:02.007857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:02.020137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 423/843] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:02.043117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf13d3e-9e79-41b2-ae5c-24a5892fcdf5 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:02.043429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf13d3e-9e79-41b2-ae5c-24a5892fcdf5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:02.043669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf13d3e-9e79-41b2-ae5c-24a5892fcdf5 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:02.045273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 424/844] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:43:02.050734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b399bcd-1161-4657-bf9f-f8d9c5891355 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:02.051015 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b399bcd-1161-4657-bf9f-f8d9c5891355 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:02.051285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b399bcd-1161-4657-bf9f-f8d9c5891355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:02.075329 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4b399bcd-1161-4657-bf9f-f8d9c5891355 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:02.077385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4b399bcd-1161-4657-bf9f-f8d9c5891355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:02.127109 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0818f8-cb4d-420f-995e-a97b1b8bc76c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:02.169064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ed547-6f2b-4d95-9b00-2c44bf799510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:02.170114 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ed547-6f2b-4d95-9b00-2c44bf799510 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:02.170574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ed547-6f2b-4d95-9b00-2c44bf799510 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:02.171027 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ed547-6f2b-4d95-9b00-2c44bf799510 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:02.177131 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 425/845] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:02.185603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ed547-6f2b-4d95-9b00-2c44bf799510 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:02.187381 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ed547-6f2b-4d95-9b00-2c44bf799510 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:02.205104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-095ed547-6f2b-4d95-9b00-2c44bf799510 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:02.205104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:02.205104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:02.205104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:02.205104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:02.205104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:02.205104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:02.205104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:02.205104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:02.205104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:02.205104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:02.205104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:02.205104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:02.205104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:02.205104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:02.205104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:02.206086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:02.206966 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:02.206966 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:02.206966 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:02.208853 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 421/846] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:02.214746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b399bcd-1161-4657-bf9f-f8d9c5891355 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:02.238187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:02.295764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ed547-6f2b-4d95-9b00-2c44bf799510 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:02.328751 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b399bcd-1161-4657-bf9f-f8d9c5891355 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:02.369237 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcaa86-ece0-49d3-8cae-3974448398dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:02.370188 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcaa86-ece0-49d3-8cae-3974448398dc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:02.370617 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcaa86-ece0-49d3-8cae-3974448398dc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:02.371035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcaa86-ece0-49d3-8cae-3974448398dc None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:02.371841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbcaa86-ece0-49d3-8cae-3974448398dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:02.372434 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbcaa86-ece0-49d3-8cae-3974448398dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:02.374110 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbcaa86-ece0-49d3-8cae-3974448398dc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:02.380340 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:02.395699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 426/847] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:02.396964 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcaa86-ece0-49d3-8cae-3974448398dc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:02.425951 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ed547-6f2b-4d95-9b00-2c44bf799510 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:02.470930 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edaed01-d756-4bb8-b9e8-60e0ca822955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:02.471565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edaed01-d756-4bb8-b9e8-60e0ca822955 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:02.471843 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edaed01-d756-4bb8-b9e8-60e0ca822955 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:02.472126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edaed01-d756-4bb8-b9e8-60e0ca822955 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:02.472706 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edaed01-d756-4bb8-b9e8-60e0ca822955 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:02.474072 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edaed01-d756-4bb8-b9e8-60e0ca822955 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-1edaed01-d756-4bb8-b9e8-60e0ca822955 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:02.482838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:02.484033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:02.484033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:02.484033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:02.484033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:02.484033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:02.484033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:02.484033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:02.484033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:02.484033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:02.484033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:02.484033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:02.484033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:02.484033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:02.484033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:02.484033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:02.484033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:02.485394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:02.485394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:02.485394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:02.485394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:02.485394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:02.485394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:02.485394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:02.485394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:02.485394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:02.485394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:02.485394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:02.485394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:02.485394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 07:43:02.485394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:02.485394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 422/848] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:43:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:02.486291 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edaed01-d756-4bb8-b9e8-60e0ca822955 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:02.498405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:02.523724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcaa86-ece0-49d3-8cae-3974448398dc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:02.558863 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f99b5-caa4-4b85-99f9-079b3c91dde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:02.559484 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f99b5-caa4-4b85-99f9-079b3c91dde5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:02.559756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f99b5-caa4-4b85-99f9-079b3c91dde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:02.560021 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f99b5-caa4-4b85-99f9-079b3c91dde5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:02.568608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f99b5-caa4-4b85-99f9-079b3c91dde5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:02.570507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f99b5-caa4-4b85-99f9-079b3c91dde5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:02.583590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-024f99b5-caa4-4b85-99f9-079b3c91dde5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 07:43:02.583590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:02.583590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:02.583590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:02.583590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:02.583590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:02.583590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:02.583590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:02.583590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:02.583590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:02.583590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:02.583590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:02.583590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:43:02.583590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:43:02.583590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:43:02.583590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:02.584577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:02.585540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:02.585540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:02.585540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:43:02.585540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:43:02.585540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:43:02.585540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:43:02.585540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:43:02.585540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:43:02.585540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 07:43:02.585540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:02.585540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 427/849] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:43:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:02.586122 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f99b5-caa4-4b85-99f9-079b3c91dde5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:02.594734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edaed01-d756-4bb8-b9e8-60e0ca822955 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:02.622906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:02.632372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1389750-da20-498d-a7f0-eb74eb818b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:02.632942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1389750-da20-498d-a7f0-eb74eb818b0b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:02.633217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1389750-da20-498d-a7f0-eb74eb818b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:02.633490 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1389750-da20-498d-a7f0-eb74eb818b0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:02.644922 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1389750-da20-498d-a7f0-eb74eb818b0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:43:02.649712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-d1389750-da20-498d-a7f0-eb74eb818b0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:43:02.649712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:02.649712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:02.649712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:02.649712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:02.649712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:02.649712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:02.649712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:02.649712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:02.649712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:02.649712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:02.649712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:02.649712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:43:02.649712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:43:02.649712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:43:02.649712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:43:02.650795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:43:02.650795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:43:02.650795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:43:02.650795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:43:02.650795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:02.650795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:02.650795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:02.650795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:02.650795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:02.650795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:02.650795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:02.650795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:02.650795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:02.650795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:02.650795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:02.650795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:02.653190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:02.653190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:02.653190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:02.653190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:02.653190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:43:02.653190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:43:02.653190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:43:02.653190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:43:02.653190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:43:02.653190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:43:02.653190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:43:02.653190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:02.653190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 423/850] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:43:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:02.653190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1389750-da20-498d-a7f0-eb74eb818b0b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:02.702086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f99b5-caa4-4b85-99f9-079b3c91dde5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:02.738362 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4c6d8-f17e-4e3a-aca4-eed75592f444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:02.738960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4c6d8-f17e-4e3a-aca4-eed75592f444 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:02.739367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4c6d8-f17e-4e3a-aca4-eed75592f444 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:02.739655 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4c6d8-f17e-4e3a-aca4-eed75592f444 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:02.740304 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4c6d8-f17e-4e3a-aca4-eed75592f444 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:02.740607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4c6d8-f17e-4e3a-aca4-eed75592f444 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:02.741963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4c6d8-f17e-4e3a-aca4-eed75592f444 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:02.746702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 428/851] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:43:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:02.747476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4c6d8-f17e-4e3a-aca4-eed75592f444 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:02.752480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:02.769341 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1389750-da20-498d-a7f0-eb74eb818b0b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:02.805373 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6f0b9-cb8b-46fe-b35f-509b17d13c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:02.806321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6f0b9-cb8b-46fe-b35f-509b17d13c4a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:02.806847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6f0b9-cb8b-46fe-b35f-509b17d13c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:02.807178 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6f0b9-cb8b-46fe-b35f-509b17d13c4a None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:02.807898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6f0b9-cb8b-46fe-b35f-509b17d13c4a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:02.808244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6f0b9-cb8b-46fe-b35f-509b17d13c4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:02.809661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6f0b9-cb8b-46fe-b35f-509b17d13c4a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:02.836242 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 424/852] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Sep 7 07:43:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:02.837294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6f0b9-cb8b-46fe-b35f-509b17d13c4a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:02.861233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4c6d8-f17e-4e3a-aca4-eed75592f444 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:02.887556 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:02.896788 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1397b39-5137-4642-ba5b-af33f5a8791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:02.897394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1397b39-5137-4642-ba5b-af33f5a8791d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:02.897672 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1397b39-5137-4642-ba5b-af33f5a8791d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:02.897944 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1397b39-5137-4642-ba5b-af33f5a8791d None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:02.898598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1397b39-5137-4642-ba5b-af33f5a8791d None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:02.898902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1397b39-5137-4642-ba5b-af33f5a8791d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:02.900301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1397b39-5137-4642-ba5b-af33f5a8791d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:02.908680 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 429/853] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:43:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:02.909450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1397b39-5137-4642-ba5b-af33f5a8791d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:02.952831 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6f0b9-cb8b-46fe-b35f-509b17d13c4a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:02.979510 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:02.991334 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42511b51-465b-416e-aeab-4eb6662a8287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:02.991942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42511b51-465b-416e-aeab-4eb6662a8287 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:02.992601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42511b51-465b-416e-aeab-4eb6662a8287 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:02.992884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42511b51-465b-416e-aeab-4eb6662a8287 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:02.993809 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42511b51-465b-416e-aeab-4eb6662a8287 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:02.994181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42511b51-465b-416e-aeab-4eb6662a8287 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:02.995731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42511b51-465b-416e-aeab-4eb6662a8287 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:03.001505 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 425/854] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:43:02 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:03.002415 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42511b51-465b-416e-aeab-4eb6662a8287 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:03.033911 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1397b39-5137-4642-ba5b-af33f5a8791d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:03.075243 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277557b-a47f-48a5-a2f4-7d9265d4e8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:03.076475 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277557b-a47f-48a5-a2f4-7d9265d4e8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:03.077052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277557b-a47f-48a5-a2f4-7d9265d4e8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:03.077564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277557b-a47f-48a5-a2f4-7d9265d4e8d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:03.145665 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42511b51-465b-416e-aeab-4eb6662a8287 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:03.160495 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277557b-a47f-48a5-a2f4-7d9265d4e8d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:03.162187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:03.163333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277557b-a47f-48a5-a2f4-7d9265d4e8d9 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:03.196986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7101c4-1557-4305-9360-330af40cf524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:03.197619 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7101c4-1557-4305-9360-330af40cf524 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:03.197894 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7101c4-1557-4305-9360-330af40cf524 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:03.198169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7101c4-1557-4305-9360-330af40cf524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:03.212898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7101c4-1557-4305-9360-330af40cf524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:03.215132 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7101c4-1557-4305-9360-330af40cf524 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:03.237470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-af7101c4-1557-4305-9360-330af40cf524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:03.237470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:03.237470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:03.237470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:03.237470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:03.237470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:03.237470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:03.237470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:03.237470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:03.237470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:03.237470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:03.237470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:03.237470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:03.237470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:03.237470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:03.237470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:03.238519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:03.239434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:03.239434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:03.239434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:03.241561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 426/855] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:03.246567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:03.247467 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7101c4-1557-4305-9360-330af40cf524 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:03.308404 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 430/856] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:43:02 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:43:03.310365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277557b-a47f-48a5-a2f4-7d9265d4e8d9 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:03.374299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7101c4-1557-4305-9360-330af40cf524 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:03.419236 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277557b-a47f-48a5-a2f4-7d9265d4e8d9 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:03.420210 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0368043-73a7-405d-9ba6-69ed3d370aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:03.420821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0368043-73a7-405d-9ba6-69ed3d370aeb None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:03.421101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0368043-73a7-405d-9ba6-69ed3d370aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:03.421375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0368043-73a7-405d-9ba6-69ed3d370aeb None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:03.422745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0368043-73a7-405d-9ba6-69ed3d370aeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:03.423302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0368043-73a7-405d-9ba6-69ed3d370aeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:03.424806 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0368043-73a7-405d-9ba6-69ed3d370aeb None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:03.442857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 427/857] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:03.444114 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0368043-73a7-405d-9ba6-69ed3d370aeb None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:03.459141 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9b149-e697-4d42-ba63-e009d194507e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:03.459844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9b149-e697-4d42-ba63-e009d194507e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:03.460307 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9b149-e697-4d42-ba63-e009d194507e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:03.460689 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9b149-e697-4d42-ba63-e009d194507e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:03.472446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9b149-e697-4d42-ba63-e009d194507e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:03.474075 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9b149-e697-4d42-ba63-e009d194507e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:03.475077 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:03.491157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-74f9b149-e697-4d42-ba63-e009d194507e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:03.491157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:03.491157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:03.491157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:03.491157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:03.491157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:03.491157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:03.491157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:03.491157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:03.491157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:03.491157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:03.491157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:03.491157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:03.491157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:03.491157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:03.491157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:03.492303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:03.493190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:03.493190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:03.493190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:03.497740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 431/858] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:03.498909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9b149-e697-4d42-ba63-e009d194507e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:03.519477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:03.569715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0368043-73a7-405d-9ba6-69ed3d370aeb None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:03.604448 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9b149-e697-4d42-ba63-e009d194507e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:03.610138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da310c-7c4e-471d-9bac-8a3fb2b15011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:03.611278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da310c-7c4e-471d-9bac-8a3fb2b15011 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:03.611783 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da310c-7c4e-471d-9bac-8a3fb2b15011 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:03.612415 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da310c-7c4e-471d-9bac-8a3fb2b15011 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:03.620552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da310c-7c4e-471d-9bac-8a3fb2b15011 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:03.622347 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da310c-7c4e-471d-9bac-8a3fb2b15011 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:03.635691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-30da310c-7c4e-471d-9bac-8a3fb2b15011 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 07:43:03.635691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:03.635691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:03.635691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:03.635691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:03.635691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:03.635691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:03.635691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:03.635691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:03.635691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:03.635691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:03.635691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:03.635691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:43:03.635691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:43:03.635691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:43:03.635691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:03.636686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:03.637675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:03.637675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:03.637675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:43:03.637675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:43:03.637675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:43:03.637675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:43:03.637675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:43:03.637675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:43:03.637675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 07:43:03.637675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:03.645842 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 428/859] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 7 07:43:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:03.646738 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da310c-7c4e-471d-9bac-8a3fb2b15011 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:03.649279 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18182b6b-3a25-4297-ac17-ec103670922d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:03.649889 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182b6b-3a25-4297-ac17-ec103670922d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:03.650166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182b6b-3a25-4297-ac17-ec103670922d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:03.650542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182b6b-3a25-4297-ac17-ec103670922d None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:03.651119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18182b6b-3a25-4297-ac17-ec103670922d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:03.651422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18182b6b-3a25-4297-ac17-ec103670922d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:03.652915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18182b6b-3a25-4297-ac17-ec103670922d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:03.670378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 432/860] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:03.671603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18182b6b-3a25-4297-ac17-ec103670922d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:03.744361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:03.755643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:03.769558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da310c-7c4e-471d-9bac-8a3fb2b15011 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:03.779243 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18182b6b-3a25-4297-ac17-ec103670922d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:03.809658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b88b1-98a7-4201-92aa-52cd4b2bd10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:03.810652 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b88b1-98a7-4201-92aa-52cd4b2bd10d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:03.811264 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b88b1-98a7-4201-92aa-52cd4b2bd10d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:03.811739 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b88b1-98a7-4201-92aa-52cd4b2bd10d None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:03.812958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b88b1-98a7-4201-92aa-52cd4b2bd10d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:03.813836 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b88b1-98a7-4201-92aa-52cd4b2bd10d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:03.815924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b88b1-98a7-4201-92aa-52cd4b2bd10d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:03.817173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af17fed-6b97-453b-a700-8688be29df3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:03.817804 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af17fed-6b97-453b-a700-8688be29df3b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:03.818081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af17fed-6b97-453b-a700-8688be29df3b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:03.818353 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af17fed-6b97-453b-a700-8688be29df3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:03.825685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af17fed-6b97-453b-a700-8688be29df3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:03.827153 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af17fed-6b97-453b-a700-8688be29df3b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:03.841457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-4af17fed-6b97-453b-a700-8688be29df3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:03.841457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:03.841457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:03.841457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:03.841457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:03.841457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:03.841457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:03.841457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:03.841457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:03.841457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:03.841457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:03.841457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:03.841457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:43:03.841457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:43:03.841457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:03.842365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:03.843254 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:43:03.843254 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:43:03.843254 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:43:03.843254 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:43:03.843254 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:43:03.843254 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:43:03.843254 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:03.843254 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:03.845287 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 429/862] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Sep 7 07:43:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:03.845426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 433/862] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:43:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:03.846158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af17fed-6b97-453b-a700-8688be29df3b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:03.850077 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b88b1-98a7-4201-92aa-52cd4b2bd10d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:03.975768 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af17fed-6b97-453b-a700-8688be29df3b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:03.978493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b88b1-98a7-4201-92aa-52cd4b2bd10d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:04.011467 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6cbda-37af-4219-b8f3-e5f4d2902fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:04.012135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6cbda-37af-4219-b8f3-e5f4d2902fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:04.012418 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6cbda-37af-4219-b8f3-e5f4d2902fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:04.012691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6cbda-37af-4219-b8f3-e5f4d2902fe6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:04.022947 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:04.024529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b0247-2e51-47c0-9f4d-5bdb2d31f53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:04.025592 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b0247-2e51-47c0-9f4d-5bdb2d31f53d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:04.026127 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b0247-2e51-47c0-9f4d-5bdb2d31f53d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:04.026767 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b0247-2e51-47c0-9f4d-5bdb2d31f53d None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:04.027701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b0247-2e51-47c0-9f4d-5bdb2d31f53d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:04.028294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b0247-2e51-47c0-9f4d-5bdb2d31f53d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:04.030854 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b0247-2e51-47c0-9f4d-5bdb2d31f53d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:04.042744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:04.049184 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 430/863] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:43:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:04.055196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b0247-2e51-47c0-9f4d-5bdb2d31f53d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:04.114290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6cbda-37af-4219-b8f3-e5f4d2902fe6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:04.116697 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6cbda-37af-4219-b8f3-e5f4d2902fe6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:04.180739 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b0247-2e51-47c0-9f4d-5bdb2d31f53d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:04.220546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92cc4b-cc4d-46a3-bd69-7ce24915046b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:04.221415 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92cc4b-cc4d-46a3-bd69-7ce24915046b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:04.221853 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92cc4b-cc4d-46a3-bd69-7ce24915046b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:04.222279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92cc4b-cc4d-46a3-bd69-7ce24915046b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:04.223158 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92cc4b-cc4d-46a3-bd69-7ce24915046b None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=3ff3bf8da732489b8b3a8fbe5aa22f21) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:04.223463 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92cc4b-cc4d-46a3-bd69-7ce24915046b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:04.224928 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92cc4b-cc4d-46a3-bd69-7ce24915046b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:04.231182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 434/864] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:43:03 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:43:04.235832 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 431/865] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:43:04 2022] GET /identity/v3/role_assignments?user.id=3ff3bf8da732489b8b3a8fbe5aa22f21&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:04.236015 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4acb2-d023-484f-8496-9b156da6f475 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:04.236015 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4acb2-d023-484f-8496-9b156da6f475 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:04.236015 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4acb2-d023-484f-8496-9b156da6f475 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:04.238839 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 435/866] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:04.240541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92cc4b-cc4d-46a3-bd69-7ce24915046b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:04.241278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cc957-0fb8-42c6-9a79-9ecbdf262d5f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:04.241836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cc957-0fb8-42c6-9a79-9ecbdf262d5f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:04.242384 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cc957-0fb8-42c6-9a79-9ecbdf262d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:04.268147 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-202cc957-0fb8-42c6-9a79-9ecbdf262d5f None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:04.269966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-202cc957-0fb8-42c6-9a79-9ecbdf262d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:04.316510 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:04.324525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:04.361049 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92cc4b-cc4d-46a3-bd69-7ce24915046b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:04.369882 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 436/867] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:04.404420 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5656659-c2c3-494f-84c6-8e5cbc7c97f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:04.405067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5656659-c2c3-494f-84c6-8e5cbc7c97f9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:04.405416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5656659-c2c3-494f-84c6-8e5cbc7c97f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:04.405695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5656659-c2c3-494f-84c6-8e5cbc7c97f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:04.417910 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5656659-c2c3-494f-84c6-8e5cbc7c97f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:04.419381 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5656659-c2c3-494f-84c6-8e5cbc7c97f9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:04.434899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-e5656659-c2c3-494f-84c6-8e5cbc7c97f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:04.434899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:04.434899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:04.434899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:04.434899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:04.434899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:04.434899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:04.434899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:04.434899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:04.434899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:04.434899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:04.434899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:04.434899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:04.434899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:04.434899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:04.434899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:04.435926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:04.437565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:04.437565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:04.437565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:04.437565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 432/868] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:04.442242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cc957-0fb8-42c6-9a79-9ecbdf262d5f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:04.545059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cc957-0fb8-42c6-9a79-9ecbdf262d5f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:04.581320 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b8b81-f5d1-4c90-93c7-acd9259ef18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:04.582290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b8b81-f5d1-4c90-93c7-acd9259ef18e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:04.582778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b8b81-f5d1-4c90-93c7-acd9259ef18e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:04.583259 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b8b81-f5d1-4c90-93c7-acd9259ef18e None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:04.584125 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b8b81-f5d1-4c90-93c7-acd9259ef18e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:04.584642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b8b81-f5d1-4c90-93c7-acd9259ef18e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:04.586207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b8b81-f5d1-4c90-93c7-acd9259ef18e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:04.599180 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 437/869] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:04.605173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5656659-c2c3-494f-84c6-8e5cbc7c97f9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:04.645040 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b8b81-f5d1-4c90-93c7-acd9259ef18e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:04.675634 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:04.726716 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5656659-c2c3-494f-84c6-8e5cbc7c97f9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:04.749367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:04.766299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b8b81-f5d1-4c90-93c7-acd9259ef18e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:04.775312 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde10a9-881b-4c41-9529-73dbe18445bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:04.775937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde10a9-881b-4c41-9529-73dbe18445bb None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:04.776257 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde10a9-881b-4c41-9529-73dbe18445bb None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:04.776504 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde10a9-881b-4c41-9529-73dbe18445bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:04.786334 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde10a9-881b-4c41-9529-73dbe18445bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:04.788364 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde10a9-881b-4c41-9529-73dbe18445bb None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:04.804196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-cdde10a9-881b-4c41-9529-73dbe18445bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:04.804196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:04.804196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:04.804196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:04.804196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:04.804196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:04.804196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:04.804196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:04.804196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:04.804196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:04.804196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:04.804196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:04.804196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:43:04.804196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:43:04.804196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:04.805118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:04.805984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:43:04.805984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:43:04.805984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:43:04.805984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:43:04.805984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:43:04.805984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:43:04.805984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:04.805984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:04.808152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 433/870] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:43:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:04.811042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde10a9-881b-4c41-9529-73dbe18445bb None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:04.817950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9dde3-2b47-42a7-af2c-903b656afe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:04.818562 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9dde3-2b47-42a7-af2c-903b656afe0b None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:04.818842 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9dde3-2b47-42a7-af2c-903b656afe0b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:04.819175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9dde3-2b47-42a7-af2c-903b656afe0b None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:04.820118 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9dde3-2b47-42a7-af2c-903b656afe0b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:04.821481 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9dde3-2b47-42a7-af2c-903b656afe0b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:04.842702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 438/871] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 7 07:43:04 2022] POST /identity/v3/services => generated 269 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:43:04.946729 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde10a9-881b-4c41-9529-73dbe18445bb None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:04.993694 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d99908-93bf-4a8a-aa07-c328bc030e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:04.994377 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d99908-93bf-4a8a-aa07-c328bc030e8f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:04.994659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d99908-93bf-4a8a-aa07-c328bc030e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:04.994931 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d99908-93bf-4a8a-aa07-c328bc030e8f None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:04.995635 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d99908-93bf-4a8a-aa07-c328bc030e8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:04.995941 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d99908-93bf-4a8a-aa07-c328bc030e8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:04.997478 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d99908-93bf-4a8a-aa07-c328bc030e8f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:05.011614 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 434/872] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:43:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:05.018507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9dde3-2b47-42a7-af2c-903b656afe0b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:05.115175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:05.131119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9dde3-2b47-42a7-af2c-903b656afe0b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:05.171707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876405c-c1e4-46ea-af47-9985b4c78c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:05.173350 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876405c-c1e4-46ea-af47-9985b4c78c71 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:05.173634 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876405c-c1e4-46ea-af47-9985b4c78c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:05.173909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876405c-c1e4-46ea-af47-9985b4c78c71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:05.174644 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876405c-c1e4-46ea-af47-9985b4c78c71 None admin] RBAC: Adding query filter params (role.id=e7960e6d957f4a1fa6547565db026f24, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=617f6479b90945d3b5a63325839afb15) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:05.174951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876405c-c1e4-46ea-af47-9985b4c78c71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:05.176395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876405c-c1e4-46ea-af47-9985b4c78c71 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:05.182794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 439/873] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:43:05 2022] GET /identity/v3/role_assignments?user.id=617f6479b90945d3b5a63325839afb15&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=e7960e6d957f4a1fa6547565db026f24 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:05.258151 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31666672-1204-48ad-b02a-3793d1f91940 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:05.258711 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31666672-1204-48ad-b02a-3793d1f91940 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:05.258711 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31666672-1204-48ad-b02a-3793d1f91940 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:05.260527 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 435/874] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:05.266750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7a846-96d1-42a8-8d0c-39c26637f5b3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:05.267069 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7a846-96d1-42a8-8d0c-39c26637f5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:05.267305 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7a846-96d1-42a8-8d0c-39c26637f5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:05.287134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4ef7a846-96d1-42a8-8d0c-39c26637f5b3 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:05.288406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4ef7a846-96d1-42a8-8d0c-39c26637f5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:05.365435 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 440/875] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:05 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:05.637632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31666672-1204-48ad-b02a-3793d1f91940 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:05.670438 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:05.746540 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31666672-1204-48ad-b02a-3793d1f91940 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:05.785291 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd66603-413e-40e3-bd70-ae653b38a72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:05.786276 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd66603-413e-40e3-bd70-ae653b38a72f None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:05.786758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd66603-413e-40e3-bd70-ae653b38a72f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:05.787238 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd66603-413e-40e3-bd70-ae653b38a72f None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:05.788100 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd66603-413e-40e3-bd70-ae653b38a72f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:05.789820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd66603-413e-40e3-bd70-ae653b38a72f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:05.806469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 436/876] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 7 07:43:05 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:43:06.559570 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111880c7-281b-440d-b1de-e2dd5875fa2a None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:06.560136 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111880c7-281b-440d-b1de-e2dd5875fa2a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:06.560136 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111880c7-281b-440d-b1de-e2dd5875fa2a None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:06.561710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 441/877] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:06.567771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96876452-c7f5-4184-ab5b-30620ad4fd1a None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:06.568246 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96876452-c7f5-4184-ab5b-30620ad4fd1a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:06.568333 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96876452-c7f5-4184-ab5b-30620ad4fd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:06.587065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-96876452-c7f5-4184-ab5b-30620ad4fd1a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:06.588281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-96876452-c7f5-4184-ab5b-30620ad4fd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:06.664208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 437/878] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:06.934515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111880c7-281b-440d-b1de-e2dd5875fa2a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:07.022876 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:07.045978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111880c7-281b-440d-b1de-e2dd5875fa2a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:07.084109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be33be-796a-4985-a583-01324d7fdd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:07.085014 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be33be-796a-4985-a583-01324d7fdd16 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:07.085453 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be33be-796a-4985-a583-01324d7fdd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:07.085874 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be33be-796a-4985-a583-01324d7fdd16 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:07.086604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be33be-796a-4985-a583-01324d7fdd16 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:07.088160 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be33be-796a-4985-a583-01324d7fdd16 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-e2be33be-796a-4985-a583-01324d7fdd16 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:07.098355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:07.099640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:07.099640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:07.099640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:07.099640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:07.099640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:07.099640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:07.099640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:07.099640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:07.099640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:07.099640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:07.099640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:07.099640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:07.099640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:07.099640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:07.099640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:07.099640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:07.100791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:07.100791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:07.100791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:07.100791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:07.100791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:07.100791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:07.100791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:07.100791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:07.100791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:07.100791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:07.100791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:07.100791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:07.100791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 07:43:07.100791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:07.103171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 442/879] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 7 07:43:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:07.112943 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96876452-c7f5-4184-ab5b-30620ad4fd1a None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:07.133309 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:07.151270 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03534f-f3b2-4417-b634-bbf972d7c5af None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:07.151819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03534f-f3b2-4417-b634-bbf972d7c5af None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:07.152275 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03534f-f3b2-4417-b634-bbf972d7c5af None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:07.154052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 443/880] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:43:07.159738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cb3ef-67ab-4629-8622-b8fccfa7951c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:07.160348 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cb3ef-67ab-4629-8622-b8fccfa7951c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:07.160826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cb3ef-67ab-4629-8622-b8fccfa7951c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:07.180365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f77cb3ef-67ab-4629-8622-b8fccfa7951c None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:07.182055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f77cb3ef-67ab-4629-8622-b8fccfa7951c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:07.233485 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96876452-c7f5-4184-ab5b-30620ad4fd1a None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:07.272107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 444/881] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 07:43:07.284560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a78db5-c47e-4240-bbb4-565fbc5fee22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:07.285172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a78db5-c47e-4240-bbb4-565fbc5fee22 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:07.285448 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a78db5-c47e-4240-bbb4-565fbc5fee22 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:07.285728 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a78db5-c47e-4240-bbb4-565fbc5fee22 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:07.286381 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a78db5-c47e-4240-bbb4-565fbc5fee22 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:07.286703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a78db5-c47e-4240-bbb4-565fbc5fee22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:07.288149 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a78db5-c47e-4240-bbb4-565fbc5fee22 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:07.293488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 438/882] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 7 07:43:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:07.299522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cb3ef-67ab-4629-8622-b8fccfa7951c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:07.381985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:07.406803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cb3ef-67ab-4629-8622-b8fccfa7951c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:07.445235 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b957c4-43b9-42c8-aca4-05d8a7ff5b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:07.445935 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b957c4-43b9-42c8-aca4-05d8a7ff5b80 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:07.446220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b957c4-43b9-42c8-aca4-05d8a7ff5b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:07.446493 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b957c4-43b9-42c8-aca4-05d8a7ff5b80 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:07.447139 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b957c4-43b9-42c8-aca4-05d8a7ff5b80 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:07.447444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b957c4-43b9-42c8-aca4-05d8a7ff5b80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:07.448883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b957c4-43b9-42c8-aca4-05d8a7ff5b80 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:07.455263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 445/883] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 7 07:43:07 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:07.546618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a78db5-c47e-4240-bbb4-565fbc5fee22 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:07.564190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f359b-379e-4622-a16e-d5a76bd103cb None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:07.564734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f359b-379e-4622-a16e-d5a76bd103cb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:07.564734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f359b-379e-4622-a16e-d5a76bd103cb None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:07.566372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 446/884] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:43:07.572575 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f3b2a-bb3d-4117-9428-826bcd2ff366 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:07.573153 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f3b2a-bb3d-4117-9428-826bcd2ff366 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:07.573673 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f3b2a-bb3d-4117-9428-826bcd2ff366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:07.597508 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8a5f3b2a-bb3d-4117-9428-826bcd2ff366 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:07.599865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8a5f3b2a-bb3d-4117-9428-826bcd2ff366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:07.650371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:07.680779 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a78db5-c47e-4240-bbb4-565fbc5fee22 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:07.695489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 447/885] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 07:43:07.725518 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd77c9-8a23-4e3f-8f8a-0cb4cba9a947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:07.726296 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd77c9-8a23-4e3f-8f8a-0cb4cba9a947 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:07.726673 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd77c9-8a23-4e3f-8f8a-0cb4cba9a947 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:07.727067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd77c9-8a23-4e3f-8f8a-0cb4cba9a947 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:07.727749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd77c9-8a23-4e3f-8f8a-0cb4cba9a947 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:07.729249 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd77c9-8a23-4e3f-8f8a-0cb4cba9a947 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:07.738759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-66fd77c9-8a23-4e3f-8f8a-0cb4cba9a947 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 07:43:07.738759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:07.738759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:07.738759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:07.738759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:07.738759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:07.738759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:07.738759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:07.738759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 07:43:07.738759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:07.738759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:07.738759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:07.738759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:07.738759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:07.738759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:07.738759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:07.739708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:07.740626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:07.740626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:07.740626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:07.740626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:07.740626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:07.740626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:07.740626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:07.740626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:07.740626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:07.740626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:07.740626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:07.740626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:07.740626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 07:43:07.740626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:07.740626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 439/886] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Sep 7 07:43:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:07.746157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f3b2a-bb3d-4117-9428-826bcd2ff366 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:07.821568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:07.855429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f3b2a-bb3d-4117-9428-826bcd2ff366 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:07.891907 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80333457-9bad-4b85-9e33-1f1678b7cdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:07.892833 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80333457-9bad-4b85-9e33-1f1678b7cdcc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:07.893322 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80333457-9bad-4b85-9e33-1f1678b7cdcc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:07.893766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80333457-9bad-4b85-9e33-1f1678b7cdcc None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:07.894562 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80333457-9bad-4b85-9e33-1f1678b7cdcc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:07.895057 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80333457-9bad-4b85-9e33-1f1678b7cdcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:07.896622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80333457-9bad-4b85-9e33-1f1678b7cdcc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:07.901964 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 448/887] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 7 07:43:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:07.907796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd77c9-8a23-4e3f-8f8a-0cb4cba9a947 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:08.001718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80333457-9bad-4b85-9e33-1f1678b7cdcc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:08.028822 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd77c9-8a23-4e3f-8f8a-0cb4cba9a947 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:08.069880 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7588d3-ed56-4fcd-bf7e-443896ad6e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:08.070493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7588d3-ed56-4fcd-bf7e-443896ad6e47 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:08.070796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7588d3-ed56-4fcd-bf7e-443896ad6e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:08.071035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7588d3-ed56-4fcd-bf7e-443896ad6e47 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:08.071718 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7588d3-ed56-4fcd-bf7e-443896ad6e47 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:08.071959 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7588d3-ed56-4fcd-bf7e-443896ad6e47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:08.073377 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7588d3-ed56-4fcd-bf7e-443896ad6e47 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:08.078878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 440/888] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 7 07:43:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:08.081395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39552e6-3c95-4494-9f6d-f921c292a236 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:08.081784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39552e6-3c95-4494-9f6d-f921c292a236 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:08.082062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39552e6-3c95-4494-9f6d-f921c292a236 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:08.084014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 441/889] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:08.084784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39552e6-3c95-4494-9f6d-f921c292a236 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:08.102125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:08.110217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80333457-9bad-4b85-9e33-1f1678b7cdcc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:08.146136 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc9082-0617-4c55-af4a-39beaecdd500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:08.146765 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc9082-0617-4c55-af4a-39beaecdd500 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:08.147034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc9082-0617-4c55-af4a-39beaecdd500 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:08.147306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc9082-0617-4c55-af4a-39beaecdd500 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:08.149299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:08.159843 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc9082-0617-4c55-af4a-39beaecdd500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:43:08.164658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-c0cc9082-0617-4c55-af4a-39beaecdd500 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 07:43:08.164658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:08.164658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:08.164658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:08.164658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:08.164658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:08.164658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:08.164658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:08.164658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:08.164658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:08.164658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:08.164658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:08.164658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:43:08.164658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:43:08.164658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:43:08.164658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:43:08.165607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:43:08.165607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:43:08.165607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:43:08.165607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:43:08.165607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:08.165607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:08.165607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:08.165607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:08.165607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:08.165607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:08.165607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:08.165607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:08.165607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:08.165607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:08.165607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:08.165607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:08.166425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:08.166425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:08.166425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:08.166425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:08.166425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:43:08.166425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:43:08.166425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:43:08.166425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:43:08.166425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:43:08.166425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:43:08.166425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 07:43:08.166425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:08.166425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 449/890] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Sep 7 07:43:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:08.167776 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15c906-9c69-42f9-8ed1-6b1c2c49eb6a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:08.167776 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15c906-9c69-42f9-8ed1-6b1c2c49eb6a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:08.167776 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15c906-9c69-42f9-8ed1-6b1c2c49eb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:08.185943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9a15c906-9c69-42f9-8ed1-6b1c2c49eb6a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:08.187136 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9a15c906-9c69-42f9-8ed1-6b1c2c49eb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:08.206453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39552e6-3c95-4494-9f6d-f921c292a236 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:08.242265 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79416d3-f31c-48f5-93b7-d62ecfded4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:08.242862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79416d3-f31c-48f5-93b7-d62ecfded4a6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:08.243138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79416d3-f31c-48f5-93b7-d62ecfded4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:08.243406 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79416d3-f31c-48f5-93b7-d62ecfded4a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:08.244114 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79416d3-f31c-48f5-93b7-d62ecfded4a6 None admin] RBAC: Adding query filter params (interface=public, service_id=b6eacebcf3bb428283aa93a45df4af52, region_id=RegionOne) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:08.244421 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79416d3-f31c-48f5-93b7-d62ecfded4a6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:08.245779 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79416d3-f31c-48f5-93b7-d62ecfded4a6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:08.253609 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 442/891] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Sep 7 07:43:08 2022] GET /identity/v3/endpoints?service_id=b6eacebcf3bb428283aa93a45df4af52&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:08.254390 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79416d3-f31c-48f5-93b7-d62ecfded4a6 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:08.269118 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 450/892] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:08 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:08.277010 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15c906-9c69-42f9-8ed1-6b1c2c49eb6a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:08.336598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:08.366591 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79416d3-f31c-48f5-93b7-d62ecfded4a6 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:08.416203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf3123-8beb-4dfb-95fa-b75b4d83d690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:08.416600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15c906-9c69-42f9-8ed1-6b1c2c49eb6a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:08.416801 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf3123-8beb-4dfb-95fa-b75b4d83d690 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:08.417076 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf3123-8beb-4dfb-95fa-b75b4d83d690 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:08.417346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf3123-8beb-4dfb-95fa-b75b4d83d690 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:08.418266 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf3123-8beb-4dfb-95fa-b75b4d83d690 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:08.418569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf3123-8beb-4dfb-95fa-b75b4d83d690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:08.428104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf3123-8beb-4dfb-95fa-b75b4d83d690 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:08.436645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 443/893] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Sep 7 07:43:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:08.438184 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf3123-8beb-4dfb-95fa-b75b4d83d690 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:08.446640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:08.459650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904c05b-293f-4deb-b706-203d118f187a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:08.460513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904c05b-293f-4deb-b706-203d118f187a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:08.460865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904c05b-293f-4deb-b706-203d118f187a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:08.461245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904c05b-293f-4deb-b706-203d118f187a None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:08.461967 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904c05b-293f-4deb-b706-203d118f187a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:08.463649 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904c05b-293f-4deb-b706-203d118f187a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:08.470272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 451/894] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 7 07:43:08 2022] GET /identity/v3/services => generated 1268 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:43:08.554931 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf3123-8beb-4dfb-95fa-b75b4d83d690 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:08.557335 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904c05b-293f-4deb-b706-203d118f187a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:08.605452 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ef6ee-6954-4f9e-8c1b-fc452251f634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:08.606018 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ef6ee-6954-4f9e-8c1b-fc452251f634 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:08.606295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ef6ee-6954-4f9e-8c1b-fc452251f634 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:08.606565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ef6ee-6954-4f9e-8c1b-fc452251f634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:08.612224 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:08.622972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ef6ee-6954-4f9e-8c1b-fc452251f634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:08.624899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ef6ee-6954-4f9e-8c1b-fc452251f634 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:08.641451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-0a2ef6ee-6954-4f9e-8c1b-fc452251f634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:08.641451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:08.641451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:08.641451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:08.641451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:08.641451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:08.641451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:08.641451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:08.641451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:08.641451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:08.641451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:08.641451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:08.641451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:08.641451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:08.641451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:08.641451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:08.642473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:08.643338 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:08.643338 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:08.643338 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:08.645540 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 444/895] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:08.653116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ef6ee-6954-4f9e-8c1b-fc452251f634 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:08.698740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904c05b-293f-4deb-b706-203d118f187a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:08.735150 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad970c4-563d-4c7c-a453-87e7b4dca93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:08.735803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad970c4-563d-4c7c-a453-87e7b4dca93b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:08.736092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad970c4-563d-4c7c-a453-87e7b4dca93b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:08.736380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad970c4-563d-4c7c-a453-87e7b4dca93b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:08.736959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad970c4-563d-4c7c-a453-87e7b4dca93b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:08.738352 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad970c4-563d-4c7c-a453-87e7b4dca93b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-fad970c4-563d-4c7c-a453-87e7b4dca93b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:08.750323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:08.751366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:08.751366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:08.751366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:08.751366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:08.751366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:08.751366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:08.751366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:08.751366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:08.751366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:08.751366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:08.751366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:08.751366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:08.751366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:08.751366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:08.751366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:08.751366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:08.752661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:08.752661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:08.752661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:08.752661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:08.752661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:08.752661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:08.752661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:08.752661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:08.752661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:08.752661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:08.752661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:08.752661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:08.752661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 07:43:08.752661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:08.752661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 452/896] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:43:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:08.757338 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad970c4-563d-4c7c-a453-87e7b4dca93b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:08.776721 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ef6ee-6954-4f9e-8c1b-fc452251f634 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:08.813099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782a988-8eb6-448e-b928-2e30be2bc13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:08.813761 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782a988-8eb6-448e-b928-2e30be2bc13f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:08.814039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782a988-8eb6-448e-b928-2e30be2bc13f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:08.814311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782a988-8eb6-448e-b928-2e30be2bc13f None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:08.814942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782a988-8eb6-448e-b928-2e30be2bc13f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:08.815242 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782a988-8eb6-448e-b928-2e30be2bc13f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:08.816672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782a988-8eb6-448e-b928-2e30be2bc13f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:08.830385 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 445/897] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:08.834772 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:08.837248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782a988-8eb6-448e-b928-2e30be2bc13f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:08.894367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:08.907073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad970c4-563d-4c7c-a453-87e7b4dca93b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:08.947222 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd100b-98fe-4ba0-bb4f-433735628e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:08.948222 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd100b-98fe-4ba0-bb4f-433735628e4a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:08.948696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd100b-98fe-4ba0-bb4f-433735628e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:08.949281 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd100b-98fe-4ba0-bb4f-433735628e4a None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:08.950103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd100b-98fe-4ba0-bb4f-433735628e4a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:08.950596 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd100b-98fe-4ba0-bb4f-433735628e4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:08.952268 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd100b-98fe-4ba0-bb4f-433735628e4a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:08.958387 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 453/898] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:43:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:08.964415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd100b-98fe-4ba0-bb4f-433735628e4a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:08.967275 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782a988-8eb6-448e-b928-2e30be2bc13f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:09.006442 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ccb13-139b-43c7-8cf4-b9d55c3cb1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:09.007158 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ccb13-139b-43c7-8cf4-b9d55c3cb1ea None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:09.007498 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ccb13-139b-43c7-8cf4-b9d55c3cb1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:09.007837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ccb13-139b-43c7-8cf4-b9d55c3cb1ea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:09.015343 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ccb13-139b-43c7-8cf4-b9d55c3cb1ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:09.016842 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ccb13-139b-43c7-8cf4-b9d55c3cb1ea None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:09.035882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-383ccb13-139b-43c7-8cf4-b9d55c3cb1ea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 07:43:09.035882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:09.035882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:09.035882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:09.035882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:09.035882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:09.035882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:09.035882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:09.035882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:09.035882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:09.035882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:09.035882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:09.035882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:43:09.035882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:43:09.035882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:43:09.035882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:09.036914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:09.037851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:09.037851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:09.037851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:43:09.037851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:43:09.037851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:43:09.037851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:43:09.037851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:43:09.037851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:43:09.037851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 07:43:09.037851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:09.041697 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 446/899] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 7 07:43:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:09.042664 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ccb13-139b-43c7-8cf4-b9d55c3cb1ea None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:09.091154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd100b-98fe-4ba0-bb4f-433735628e4a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:09.109034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:09.129513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bda61-1c08-4367-9213-c0dc083bfcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:09.130112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bda61-1c08-4367-9213-c0dc083bfcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:09.130389 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bda61-1c08-4367-9213-c0dc083bfcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:09.130656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bda61-1c08-4367-9213-c0dc083bfcd8 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:09.131499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bda61-1c08-4367-9213-c0dc083bfcd8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:09.131608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bda61-1c08-4367-9213-c0dc083bfcd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:09.133218 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bda61-1c08-4367-9213-c0dc083bfcd8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:09.138317 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 454/900] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:43:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:09.148184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bda61-1c08-4367-9213-c0dc083bfcd8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:09.160165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ccb13-139b-43c7-8cf4-b9d55c3cb1ea None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:09.169738 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:09.200117 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b909df-47cf-4e89-bb20-1630cd6dd7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:09.200769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b909df-47cf-4e89-bb20-1630cd6dd7b1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:09.201043 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b909df-47cf-4e89-bb20-1630cd6dd7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:09.201313 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b909df-47cf-4e89-bb20-1630cd6dd7b1 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:09.202039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b909df-47cf-4e89-bb20-1630cd6dd7b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:09.202402 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b909df-47cf-4e89-bb20-1630cd6dd7b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:09.204105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b909df-47cf-4e89-bb20-1630cd6dd7b1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:09.223072 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 447/901] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Sep 7 07:43:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:09.228517 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b909df-47cf-4e89-bb20-1630cd6dd7b1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:09.262557 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bda61-1c08-4367-9213-c0dc083bfcd8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:09.301580 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384095ae-a22a-4429-8b07-8644cdf00292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:09.302182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384095ae-a22a-4429-8b07-8644cdf00292 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:09.302513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384095ae-a22a-4429-8b07-8644cdf00292 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:09.302783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384095ae-a22a-4429-8b07-8644cdf00292 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:09.303521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384095ae-a22a-4429-8b07-8644cdf00292 None admin] RBAC: Adding query filter params (interface=public, service_id=5d3a338a05cb4ee7ada0bc615df6834b, region_id=RegionOne) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:09.303827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384095ae-a22a-4429-8b07-8644cdf00292 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:09.305203 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384095ae-a22a-4429-8b07-8644cdf00292 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:09.311168 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 455/902] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Sep 7 07:43:09 2022] GET /identity/v3/endpoints?service_id=5d3a338a05cb4ee7ada0bc615df6834b&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:09.318255 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384095ae-a22a-4429-8b07-8644cdf00292 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:09.341797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b909df-47cf-4e89-bb20-1630cd6dd7b1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:09.383446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:09.396532 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c935f4a-edf8-4d69-9988-80a559c89c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:09.397649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c935f4a-edf8-4d69-9988-80a559c89c8b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:09.398182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c935f4a-edf8-4d69-9988-80a559c89c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:09.398719 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c935f4a-edf8-4d69-9988-80a559c89c8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:09.444859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384095ae-a22a-4429-8b07-8644cdf00292 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:09.446600 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:09.480325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5678f7-436d-45eb-90d3-f869d70c4ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:09.480917 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5678f7-436d-45eb-90d3-f869d70c4ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:09.481191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5678f7-436d-45eb-90d3-f869d70c4ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:09.481517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5678f7-436d-45eb-90d3-f869d70c4ec6 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:09.482042 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5678f7-436d-45eb-90d3-f869d70c4ec6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:09.483393 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5678f7-436d-45eb-90d3-f869d70c4ec6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:09.495182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 456/903] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 7 07:43:09 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:43:09.495371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c935f4a-edf8-4d69-9988-80a559c89c8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:09.495371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c935f4a-edf8-4d69-9988-80a559c89c8b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:09.610882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 448/904] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:43:09 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 07:43:09.617551 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5678f7-436d-45eb-90d3-f869d70c4ec6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:09.704335 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb78963-8981-42be-8f99-2a200afb1924 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:09.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb78963-8981-42be-8f99-2a200afb1924 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:09.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb78963-8981-42be-8f99-2a200afb1924 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:09.706872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 449/905] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:09.711981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35302689-3509-41a6-8d6e-a53e93cb1f3c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:09.712348 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35302689-3509-41a6-8d6e-a53e93cb1f3c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:09.712558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35302689-3509-41a6-8d6e-a53e93cb1f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:09.731664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5678f7-436d-45eb-90d3-f869d70c4ec6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:09.734818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-35302689-3509-41a6-8d6e-a53e93cb1f3c None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:09.736113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-35302689-3509-41a6-8d6e-a53e93cb1f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:09.742910 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:09.776611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b704dc-3733-4aad-a081-2eb48ecc7e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:09.777712 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b704dc-3733-4aad-a081-2eb48ecc7e59 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:09.778219 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b704dc-3733-4aad-a081-2eb48ecc7e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:09.778728 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b704dc-3733-4aad-a081-2eb48ecc7e59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:09.791131 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b704dc-3733-4aad-a081-2eb48ecc7e59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:09.793166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b704dc-3733-4aad-a081-2eb48ecc7e59 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:09.803196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:09.810823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-03b704dc-3733-4aad-a081-2eb48ecc7e59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:09.810823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:09.810823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:09.810823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:09.810823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:09.810823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:09.810823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:09.810823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:09.810823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:09.810823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:09.810823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:09.810823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:09.810823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:09.810823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:09.810823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:09.810823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:09.813086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:09.814190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:09.814190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:09.814190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:09.814764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 450/906] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:09 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 07:43:09.814847 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 457/907] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:09.821897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35302689-3509-41a6-8d6e-a53e93cb1f3c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:09.925583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35302689-3509-41a6-8d6e-a53e93cb1f3c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:09.962024 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde12eb4-1bfe-4790-9191-453d5c898ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:09.962552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde12eb4-1bfe-4790-9191-453d5c898ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:09.962831 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde12eb4-1bfe-4790-9191-453d5c898ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:09.963104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde12eb4-1bfe-4790-9191-453d5c898ce5 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:09.963739 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde12eb4-1bfe-4790-9191-453d5c898ce5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:09.964043 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde12eb4-1bfe-4790-9191-453d5c898ce5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:09.965441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde12eb4-1bfe-4790-9191-453d5c898ce5 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:09.977870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 451/908] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:09.985460 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b704dc-3733-4aad-a081-2eb48ecc7e59 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:10.087678 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b704dc-3733-4aad-a081-2eb48ecc7e59 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:10.090612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde12eb4-1bfe-4790-9191-453d5c898ce5 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:10.126889 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d9a15-e498-4703-9fb5-6377e99e577c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:10.127632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d9a15-e498-4703-9fb5-6377e99e577c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:10.127904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d9a15-e498-4703-9fb5-6377e99e577c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:10.128198 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d9a15-e498-4703-9fb5-6377e99e577c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:10.136662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d9a15-e498-4703-9fb5-6377e99e577c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:10.138546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d9a15-e498-4703-9fb5-6377e99e577c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:10.152344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-2e9d9a15-e498-4703-9fb5-6377e99e577c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:10.152344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:10.152344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:10.152344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:10.152344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:10.152344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:10.152344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:10.152344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:10.152344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:10.152344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:10.152344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:10.152344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:10.152344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:43:10.152344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:43:10.152344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:10.153456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:10.154539 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:43:10.154539 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:43:10.154539 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:43:10.154539 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:43:10.154539 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:43:10.154539 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:43:10.154539 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:10.154539 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:10.156188 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 458/909] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:43:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:10.161858 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d9a15-e498-4703-9fb5-6377e99e577c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:10.162414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:10.187585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:10.221572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde12eb4-1bfe-4790-9191-453d5c898ce5 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:10.273134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb69a4f-527f-4ac5-be4f-d96a71ac3e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:10.273899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb69a4f-527f-4ac5-be4f-d96a71ac3e2a None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:10.274268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb69a4f-527f-4ac5-be4f-d96a71ac3e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:10.274636 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb69a4f-527f-4ac5-be4f-d96a71ac3e2a None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:10.275387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb69a4f-527f-4ac5-be4f-d96a71ac3e2a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:10.276891 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb69a4f-527f-4ac5-be4f-d96a71ac3e2a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:10.286631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d9a15-e498-4703-9fb5-6377e99e577c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:10.296228 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 452/910] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 7 07:43:10 2022] POST /identity/v3/services => generated 241 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:43:10.339569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5001e99-8418-4e88-9271-4b028e9472c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:10.340528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5001e99-8418-4e88-9271-4b028e9472c0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:10.340946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5001e99-8418-4e88-9271-4b028e9472c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:10.341413 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5001e99-8418-4e88-9271-4b028e9472c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:10.342388 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5001e99-8418-4e88-9271-4b028e9472c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:10.342835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5001e99-8418-4e88-9271-4b028e9472c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:10.344607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5001e99-8418-4e88-9271-4b028e9472c0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:10.358871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 459/911] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:43:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:10.369282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb69a4f-527f-4ac5-be4f-d96a71ac3e2a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:10.494731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb69a4f-527f-4ac5-be4f-d96a71ac3e2a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:10.514197 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:10.541519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305867f-8906-4e6d-b3db-9e4790826be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:10.542124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305867f-8906-4e6d-b3db-9e4790826be5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:10.542398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305867f-8906-4e6d-b3db-9e4790826be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:10.542666 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305867f-8906-4e6d-b3db-9e4790826be5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:10.543379 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305867f-8906-4e6d-b3db-9e4790826be5 None admin] RBAC: Adding query filter params (role.id=35552dee8978468c86e59379516db233, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=617f6479b90945d3b5a63325839afb15) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:10.543681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305867f-8906-4e6d-b3db-9e4790826be5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:10.545170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305867f-8906-4e6d-b3db-9e4790826be5 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:10.554143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 453/912] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:43:10 2022] GET /identity/v3/role_assignments?user.id=617f6479b90945d3b5a63325839afb15&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=35552dee8978468c86e59379516db233 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:10.749371 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6e127-def4-4476-821a-16d1cc5ed76c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:10.750466 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6e127-def4-4476-821a-16d1cc5ed76c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:10.750591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6e127-def4-4476-821a-16d1cc5ed76c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:10.751835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 460/913] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:10.757565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e458c4-7f14-4a3f-9398-b79854279297 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:10.758030 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e458c4-7f14-4a3f-9398-b79854279297 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:10.758108 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e458c4-7f14-4a3f-9398-b79854279297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:10.777603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-41e458c4-7f14-4a3f-9398-b79854279297 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:10.778919 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-41e458c4-7f14-4a3f-9398-b79854279297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:10.850873 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 454/914] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 07:43:11.132307 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6e127-def4-4476-821a-16d1cc5ed76c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:11.214587 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:11.239462 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6e127-def4-4476-821a-16d1cc5ed76c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:11.274715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5503a-d9af-44f2-be63-aeec9a6db878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:11.275348 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5503a-d9af-44f2-be63-aeec9a6db878 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:11.275619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5503a-d9af-44f2-be63-aeec9a6db878 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:11.275889 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5503a-d9af-44f2-be63-aeec9a6db878 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:11.276499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5503a-d9af-44f2-be63-aeec9a6db878 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:11.277912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5503a-d9af-44f2-be63-aeec9a6db878 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:11.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-94d5503a-d9af-44f2-be63-aeec9a6db878 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 07:43:11.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:11.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:11.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:11.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:11.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:11.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:11.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:11.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 07:43:11.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:11.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:11.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:11.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:11.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:11.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:11.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:11.288642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:11.289530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:11.289530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:11.289530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:11.289530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:11.289530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:11.289530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:11.289530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:11.289530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:11.289530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:11.289530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:11.289530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:11.289530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:11.289530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 07:43:11.289530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:11.289530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 461/915] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Sep 7 07:43:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:11.295203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e458c4-7f14-4a3f-9398-b79854279297 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:11.403751 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e458c4-7f14-4a3f-9398-b79854279297 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:11.416411 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:11.451288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82290263-df06-44c7-bfba-08dc5a98c5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:11.451903 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82290263-df06-44c7-bfba-08dc5a98c5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:11.452210 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82290263-df06-44c7-bfba-08dc5a98c5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:11.452488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82290263-df06-44c7-bfba-08dc5a98c5e2 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:11.453236 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82290263-df06-44c7-bfba-08dc5a98c5e2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:11.453930 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82290263-df06-44c7-bfba-08dc5a98c5e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:11.455362 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82290263-df06-44c7-bfba-08dc5a98c5e2 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:11.460878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 455/916] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 7 07:43:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:11.466343 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5503a-d9af-44f2-be63-aeec9a6db878 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:11.570201 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5503a-d9af-44f2-be63-aeec9a6db878 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:11.606913 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70abfdd-204c-4d10-ac7d-7a6085aeef56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:11.607740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70abfdd-204c-4d10-ac7d-7a6085aeef56 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:11.608188 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70abfdd-204c-4d10-ac7d-7a6085aeef56 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:11.608630 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70abfdd-204c-4d10-ac7d-7a6085aeef56 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:11.609464 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70abfdd-204c-4d10-ac7d-7a6085aeef56 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:11.609910 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70abfdd-204c-4d10-ac7d-7a6085aeef56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:11.611453 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70abfdd-204c-4d10-ac7d-7a6085aeef56 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:11.616835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 462/917] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 7 07:43:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:11.622620 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82290263-df06-44c7-bfba-08dc5a98c5e2 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:11.732645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82290263-df06-44c7-bfba-08dc5a98c5e2 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:11.775007 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0d877-c2b2-4309-8fb9-2e896b9d73c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:11.775627 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0d877-c2b2-4309-8fb9-2e896b9d73c5 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:11.775877 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0d877-c2b2-4309-8fb9-2e896b9d73c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:11.776181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0d877-c2b2-4309-8fb9-2e896b9d73c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:11.776862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a0d877-c2b2-4309-8fb9-2e896b9d73c5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:11.778237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a0d877-c2b2-4309-8fb9-2e896b9d73c5 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:11.801997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 456/918] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:43:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:43:11.862601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a591ed-f830-4c7c-990a-feb77c5b496d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:11.863138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a591ed-f830-4c7c-990a-feb77c5b496d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:11.863254 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a591ed-f830-4c7c-990a-feb77c5b496d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:11.864926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 463/919] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:11.870742 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32547da-1423-4cac-a524-c25f8d3594cd None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:11.871034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32547da-1423-4cac-a524-c25f8d3594cd None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:11.871300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32547da-1423-4cac-a524-c25f8d3594cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:11.893804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a32547da-1423-4cac-a524-c25f8d3594cd None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:11.895104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a32547da-1423-4cac-a524-c25f8d3594cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:11.925154 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:11.982183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 457/920] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:12.264570 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a591ed-f830-4c7c-990a-feb77c5b496d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:12.270319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:12.388577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a591ed-f830-4c7c-990a-feb77c5b496d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:12.433056 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededa1f8-0aea-4da0-99ab-8c203b235887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:12.433695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededa1f8-0aea-4da0-99ab-8c203b235887 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:12.433972 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededa1f8-0aea-4da0-99ab-8c203b235887 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:12.434245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededa1f8-0aea-4da0-99ab-8c203b235887 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:12.434831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ededa1f8-0aea-4da0-99ab-8c203b235887 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:12.436224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ededa1f8-0aea-4da0-99ab-8c203b235887 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ededa1f8-0aea-4da0-99ab-8c203b235887 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:12.446444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:12.447438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:12.447438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:12.447438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:12.447438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:12.447438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:12.447438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:12.447438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:12.447438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:12.447438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:12.447438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:12.447438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:12.447438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:12.447438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:12.447438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:12.447438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:12.447438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:12.448301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:12.448301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:12.448301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:12.448301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:12.448301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:12.448301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:12.448301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:12.448301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:12.448301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:12.448301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:12.448301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:12.448301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:12.448301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 07:43:12.448301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:12.452222 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 464/921] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:43:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:12.454893 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32547da-1423-4cac-a524-c25f8d3594cd None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:12.563732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32547da-1423-4cac-a524-c25f8d3594cd None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:12.606019 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594a751-8eaa-44b3-9b1f-9631260dac45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:12.606650 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594a751-8eaa-44b3-9b1f-9631260dac45 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:12.606928 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594a751-8eaa-44b3-9b1f-9631260dac45 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:12.607201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594a751-8eaa-44b3-9b1f-9631260dac45 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:12.607850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594a751-8eaa-44b3-9b1f-9631260dac45 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:12.608190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594a751-8eaa-44b3-9b1f-9631260dac45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:12.610428 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594a751-8eaa-44b3-9b1f-9631260dac45 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:12.621383 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 458/922] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:43:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:12.622983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededa1f8-0aea-4da0-99ab-8c203b235887 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:12.689022 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b516852-903f-423f-bd62-4d6e1cf93cf5 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:12.689525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b516852-903f-423f-bd62-4d6e1cf93cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:12.689801 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b516852-903f-423f-bd62-4d6e1cf93cf5 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:12.692237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 459/923] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:12.698539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33543d68-a35d-4353-8d7e-0f4821b0e07e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:12.698822 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33543d68-a35d-4353-8d7e-0f4821b0e07e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:12.699092 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33543d68-a35d-4353-8d7e-0f4821b0e07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:12.709933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:12.719168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-33543d68-a35d-4353-8d7e-0f4821b0e07e None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:12.720383 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-33543d68-a35d-4353-8d7e-0f4821b0e07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:12.736917 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:12.752513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededa1f8-0aea-4da0-99ab-8c203b235887 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:12.795904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e10efa-2a55-4714-9603-e8b35c756865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:12.796537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e10efa-2a55-4714-9603-e8b35c756865 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:12.797408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e10efa-2a55-4714-9603-e8b35c756865 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:12.797676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e10efa-2a55-4714-9603-e8b35c756865 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:12.798347 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e10efa-2a55-4714-9603-e8b35c756865 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:12.798647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e10efa-2a55-4714-9603-e8b35c756865 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:12.804257 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 460/924] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 07:43:12.804723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e10efa-2a55-4714-9603-e8b35c756865 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:12.804819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be72efa-858f-4b63-8e80-0890406d6bf6 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:12.804819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be72efa-858f-4b63-8e80-0890406d6bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:12.804819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be72efa-858f-4b63-8e80-0890406d6bf6 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:12.804819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 461/925] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:12.807476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a9a14-274a-4aed-af17-81c3802a28d8 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:12.807749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a9a14-274a-4aed-af17-81c3802a28d8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:12.808013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a9a14-274a-4aed-af17-81c3802a28d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:12.815489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 465/926] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 7 07:43:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:12.821545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e10efa-2a55-4714-9603-e8b35c756865 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:12.832760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-416a9a14-274a-4aed-af17-81c3802a28d8 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:12.834945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-416a9a14-274a-4aed-af17-81c3802a28d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:12.920427 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 462/927] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 07:43:12.942199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e10efa-2a55-4714-9603-e8b35c756865 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:12.973785 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:12.980021 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049f21f-dc51-45a8-ac18-e04bbc65d8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:12.980647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049f21f-dc51-45a8-ac18-e04bbc65d8f7 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:12.980923 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049f21f-dc51-45a8-ac18-e04bbc65d8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:12.981190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049f21f-dc51-45a8-ac18-e04bbc65d8f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:12.981846 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049f21f-dc51-45a8-ac18-e04bbc65d8f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:12.983199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049f21f-dc51-45a8-ac18-e04bbc65d8f7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:13.006879 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 466/928] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:43:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:43:13.084364 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a9a14-274a-4aed-af17-81c3802a28d8 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:13.180018 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:13.201899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a9a14-274a-4aed-af17-81c3802a28d8 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:13.210503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049f21f-dc51-45a8-ac18-e04bbc65d8f7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:13.249280 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0178652-771d-4e1b-ad5d-9098c9adcf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:13.250233 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0178652-771d-4e1b-ad5d-9098c9adcf21 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:13.250508 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0178652-771d-4e1b-ad5d-9098c9adcf21 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:13.250772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0178652-771d-4e1b-ad5d-9098c9adcf21 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:13.251519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0178652-771d-4e1b-ad5d-9098c9adcf21 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:13.252966 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0178652-771d-4e1b-ad5d-9098c9adcf21 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-a0178652-771d-4e1b-ad5d-9098c9adcf21 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:13.262048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:13.263040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:13.263040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:13.263040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:13.263040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:13.263040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:13.263040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:13.263040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:13.263040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:13.263040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:13.263040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:13.263040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:13.263040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:13.263040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:13.263040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:13.263040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:13.263040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:13.263862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:13.263862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:13.263862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:13.263862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:13.263862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:13.263862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:13.263862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:13.263862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:13.263862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:13.263862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:13.263862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:13.263862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:13.263862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 07:43:13.263862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:13.263862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 463/929] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 7 07:43:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:13.270778 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0178652-771d-4e1b-ad5d-9098c9adcf21 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:13.343059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049f21f-dc51-45a8-ac18-e04bbc65d8f7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:13.385578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0178652-771d-4e1b-ad5d-9098c9adcf21 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:13.387952 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4c09f-bc59-4616-8b0e-02fa24ef61bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:13.389499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4c09f-bc59-4616-8b0e-02fa24ef61bb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:13.389778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4c09f-bc59-4616-8b0e-02fa24ef61bb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:13.390051 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4c09f-bc59-4616-8b0e-02fa24ef61bb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:13.403821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4c09f-bc59-4616-8b0e-02fa24ef61bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:43:13.411544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f0d4c09f-bc59-4616-8b0e-02fa24ef61bb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 07:43:13.411544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:13.411544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:13.411544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:13.411544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:13.411544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:13.411544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:13.411544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:13.411544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:13.411544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:13.411544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:13.411544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:13.411544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:43:13.411544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:43:13.411544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:43:13.411544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:43:13.412515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:43:13.412515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:43:13.412515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:43:13.412515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:43:13.412515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:13.412515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:13.412515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:13.412515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:13.412515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:13.412515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:13.412515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:13.412515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:13.412515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:13.412515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:13.412515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:13.412515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:13.413458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:13.413458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:13.413458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:13.413458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:13.413458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:43:13.413458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:43:13.413458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:43:13.413458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:43:13.413458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:43:13.413458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:43:13.413458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 07:43:13.413458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:13.413458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 467/930] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Sep 7 07:43:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:13.421376 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4c09f-bc59-4616-8b0e-02fa24ef61bb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:13.429229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81858304-1e66-4086-9dda-b8636a74952a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:13.430068 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81858304-1e66-4086-9dda-b8636a74952a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:13.430541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81858304-1e66-4086-9dda-b8636a74952a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:13.430993 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81858304-1e66-4086-9dda-b8636a74952a None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:13.431881 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81858304-1e66-4086-9dda-b8636a74952a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:13.432397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81858304-1e66-4086-9dda-b8636a74952a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:13.433956 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81858304-1e66-4086-9dda-b8636a74952a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:13.443988 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 464/931] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 7 07:43:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:13.445735 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81858304-1e66-4086-9dda-b8636a74952a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:13.449700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:13.475218 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:13.531639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4c09f-bc59-4616-8b0e-02fa24ef61bb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:13.557118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81858304-1e66-4086-9dda-b8636a74952a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:13.567194 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d830528-969e-4eee-8e2d-ecc6ebd590c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:13.568188 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d830528-969e-4eee-8e2d-ecc6ebd590c8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:13.568188 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d830528-969e-4eee-8e2d-ecc6ebd590c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:13.568556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d830528-969e-4eee-8e2d-ecc6ebd590c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:13.569276 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d830528-969e-4eee-8e2d-ecc6ebd590c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:13.569894 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d830528-969e-4eee-8e2d-ecc6ebd590c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:13.571390 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d830528-969e-4eee-8e2d-ecc6ebd590c8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:13.580344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 468/932] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Sep 7 07:43:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:13.589297 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d830528-969e-4eee-8e2d-ecc6ebd590c8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:13.600943 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce7526-2930-4302-a1de-ed20124b069f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:13.601468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce7526-2930-4302-a1de-ed20124b069f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:13.601796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce7526-2930-4302-a1de-ed20124b069f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:13.602070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce7526-2930-4302-a1de-ed20124b069f None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:13.602796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce7526-2930-4302-a1de-ed20124b069f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:13.603098 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce7526-2930-4302-a1de-ed20124b069f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:13.604492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce7526-2930-4302-a1de-ed20124b069f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:13.610488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 465/933] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 7 07:43:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:13.616398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce7526-2930-4302-a1de-ed20124b069f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:13.692454 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:13.710497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d830528-969e-4eee-8e2d-ecc6ebd590c8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:13.725339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:13.731702 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce7526-2930-4302-a1de-ed20124b069f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:13.756845 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781bd8b-d415-4b55-bd6a-300cba3022bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:13.757481 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781bd8b-d415-4b55-bd6a-300cba3022bf None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:13.757756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781bd8b-d415-4b55-bd6a-300cba3022bf None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:13.758024 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781bd8b-d415-4b55-bd6a-300cba3022bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:13.770851 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781bd8b-d415-4b55-bd6a-300cba3022bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:13.772510 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5119ae-9d46-42cc-b92a-5548156ff49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:13.772643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781bd8b-d415-4b55-bd6a-300cba3022bf None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:13.773459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5119ae-9d46-42cc-b92a-5548156ff49b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:13.773955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5119ae-9d46-42cc-b92a-5548156ff49b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:13.774439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5119ae-9d46-42cc-b92a-5548156ff49b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:13.775329 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5119ae-9d46-42cc-b92a-5548156ff49b None admin] RBAC: Adding query filter params (interface=public, service_id=225f0761e1054002be68243760751f9f, region_id=RegionOne) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:13.775881 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5119ae-9d46-42cc-b92a-5548156ff49b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:13.777625 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5119ae-9d46-42cc-b92a-5548156ff49b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:13.784257 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 466/934] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Sep 7 07:43:13 2022] GET /identity/v3/endpoints?service_id=225f0761e1054002be68243760751f9f&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:13.788903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-9781bd8b-d415-4b55-bd6a-300cba3022bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:13.788903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:13.788903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:13.788903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:13.788903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:13.788903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:13.788903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:13.788903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:13.788903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:13.788903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:13.788903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:13.788903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:13.788903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:13.788903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:13.788903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:13.788903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:13.789898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:13.790779 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:13.790779 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:13.790779 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:13.790944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5119ae-9d46-42cc-b92a-5548156ff49b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:13.798026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 469/935] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:13.799634 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781bd8b-d415-4b55-bd6a-300cba3022bf None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:13.908664 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5119ae-9d46-42cc-b92a-5548156ff49b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:13.935003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781bd8b-d415-4b55-bd6a-300cba3022bf None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:13.950667 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:13.959221 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fcaad9-df15-43bb-a2fd-222bb9a9693c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:13.960287 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcaad9-df15-43bb-a2fd-222bb9a9693c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:13.961002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcaad9-df15-43bb-a2fd-222bb9a9693c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:13.961431 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcaad9-df15-43bb-a2fd-222bb9a9693c None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:13.962520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fcaad9-df15-43bb-a2fd-222bb9a9693c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:13.965052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fcaad9-df15-43bb-a2fd-222bb9a9693c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:13.974700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 467/936] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 7 07:43:13 2022] GET /identity/v3/services => generated 1497 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:43:13.982164 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aea6d1-2d48-4a78-b37f-b6a5c22281dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:13.983504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aea6d1-2d48-4a78-b37f-b6a5c22281dc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:13.984539 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aea6d1-2d48-4a78-b37f-b6a5c22281dc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:13.985022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aea6d1-2d48-4a78-b37f-b6a5c22281dc None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:13.985997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aea6d1-2d48-4a78-b37f-b6a5c22281dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:13.986566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aea6d1-2d48-4a78-b37f-b6a5c22281dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:13.988280 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aea6d1-2d48-4a78-b37f-b6a5c22281dc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:14.003278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 470/937] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:14.009419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fcaad9-df15-43bb-a2fd-222bb9a9693c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:14.131243 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fcaad9-df15-43bb-a2fd-222bb9a9693c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:14.170745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b7a9f-e017-4add-96e4-c7d5a415874b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:14.171109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac5549-a471-4202-b06a-58d9f26e2495 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:14.171397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac5549-a471-4202-b06a-58d9f26e2495 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:14.171642 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b7a9f-e017-4add-96e4-c7d5a415874b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:14.171759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac5549-a471-4202-b06a-58d9f26e2495 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:14.171917 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b7a9f-e017-4add-96e4-c7d5a415874b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:14.172233 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b7a9f-e017-4add-96e4-c7d5a415874b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:14.173739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 471/938] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:43:14.179670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28250dc5-9885-4b18-a8ba-683e69d0faf7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:14.180038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28250dc5-9885-4b18-a8ba-683e69d0faf7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:14.180283 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28250dc5-9885-4b18-a8ba-683e69d0faf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:14.181193 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b7a9f-e017-4add-96e4-c7d5a415874b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:14.182636 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b7a9f-e017-4add-96e4-c7d5a415874b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:14.197207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-5a9b7a9f-e017-4add-96e4-c7d5a415874b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 07:43:14.197207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:14.197207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:14.197207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:14.197207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:14.197207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:14.197207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:14.197207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:14.197207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:14.197207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:14.197207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:14.197207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:14.197207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:43:14.197207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:43:14.197207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:43:14.197207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:14.198177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:14.199419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:14.199419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:14.199419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:43:14.199419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:43:14.199419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:43:14.199419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:43:14.199419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:43:14.199419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:43:14.199419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 07:43:14.199419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:14.199419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 468/939] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 7 07:43:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:14.203503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-28250dc5-9885-4b18-a8ba-683e69d0faf7 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:14.205053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b7a9f-e017-4add-96e4-c7d5a415874b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:14.206108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-28250dc5-9885-4b18-a8ba-683e69d0faf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:14.225048 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:14.285524 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:14.286774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 472/940] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:14.333411 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b7a9f-e017-4add-96e4-c7d5a415874b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:14.368821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a8529-9d01-48f4-8e6c-fcbf86bca36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:14.369428 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a8529-9d01-48f4-8e6c-fcbf86bca36f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:14.369704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a8529-9d01-48f4-8e6c-fcbf86bca36f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:14.369973 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a8529-9d01-48f4-8e6c-fcbf86bca36f None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:14.370692 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a8529-9d01-48f4-8e6c-fcbf86bca36f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:14.370995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a8529-9d01-48f4-8e6c-fcbf86bca36f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:14.372451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a8529-9d01-48f4-8e6c-fcbf86bca36f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:14.391983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 469/941] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Sep 7 07:43:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:14.397621 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28250dc5-9885-4b18-a8ba-683e69d0faf7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:14.505674 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28250dc5-9885-4b18-a8ba-683e69d0faf7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:14.541082 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dfe93-f941-4bef-b490-19f0f98baf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:14.541718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dfe93-f941-4bef-b490-19f0f98baf1f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:14.541993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dfe93-f941-4bef-b490-19f0f98baf1f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:14.542267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dfe93-f941-4bef-b490-19f0f98baf1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:14.552609 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78dfe93-f941-4bef-b490-19f0f98baf1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:14.554039 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78dfe93-f941-4bef-b490-19f0f98baf1f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:14.569600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d78dfe93-f941-4bef-b490-19f0f98baf1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:14.569600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:14.569600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:14.569600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:14.569600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:14.569600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:14.569600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:14.569600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:14.569600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:14.569600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:14.569600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:14.569600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:14.569600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:14.569600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:14.569600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:14.569600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:14.570540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:14.571659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:14.571659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:14.571659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:14.571659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 473/942] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:14.576828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a8529-9d01-48f4-8e6c-fcbf86bca36f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:14.585104 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dfe93-f941-4bef-b490-19f0f98baf1f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:14.589639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:14.681990 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a8529-9d01-48f4-8e6c-fcbf86bca36f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:14.696343 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dfe93-f941-4bef-b490-19f0f98baf1f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:14.697646 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:14.719341 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a68adb-4a54-4ab0-8f9c-9d6bafedc17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:14.720160 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a68adb-4a54-4ab0-8f9c-9d6bafedc17e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:14.720512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a68adb-4a54-4ab0-8f9c-9d6bafedc17e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:14.720854 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a68adb-4a54-4ab0-8f9c-9d6bafedc17e None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:14.721578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a68adb-4a54-4ab0-8f9c-9d6bafedc17e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:14.721949 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a68adb-4a54-4ab0-8f9c-9d6bafedc17e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:14.723377 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a68adb-4a54-4ab0-8f9c-9d6bafedc17e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:14.736563 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47e1c6-ff4f-4f9f-aae8-cc1525495269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:14.736776 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 470/943] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:14.737262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47e1c6-ff4f-4f9f-aae8-cc1525495269 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:14.737541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47e1c6-ff4f-4f9f-aae8-cc1525495269 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:14.737907 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47e1c6-ff4f-4f9f-aae8-cc1525495269 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:14.738489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47e1c6-ff4f-4f9f-aae8-cc1525495269 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:14.739843 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47e1c6-ff4f-4f9f-aae8-cc1525495269 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:14.742444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a68adb-4a54-4ab0-8f9c-9d6bafedc17e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-7a47e1c6-ff4f-4f9f-aae8-cc1525495269 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:14.752600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:14.753648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:14.753648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:14.753648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:14.753648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:14.753648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:14.753648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:14.753648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:14.753648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:14.753648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:14.753648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:14.753648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:14.753648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:14.753648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:14.753648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:14.753648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:14.753648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:14.755034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:14.755034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:14.755034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:14.755034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:14.755034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:14.755034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:14.755034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:14.755034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:14.755034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:14.755034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:14.755034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:14.755034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:14.755034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 07:43:14.755034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:14.755034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 474/944] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:43:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:14.759505 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47e1c6-ff4f-4f9f-aae8-cc1525495269 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:14.844678 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:14.873597 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a68adb-4a54-4ab0-8f9c-9d6bafedc17e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:14.892699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47e1c6-ff4f-4f9f-aae8-cc1525495269 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:14.913917 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7851d366-97a7-41af-8d2b-7e7ee1761b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:14.914569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851d366-97a7-41af-8d2b-7e7ee1761b72 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:14.914877 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851d366-97a7-41af-8d2b-7e7ee1761b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:14.915153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851d366-97a7-41af-8d2b-7e7ee1761b72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:14.923133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7851d366-97a7-41af-8d2b-7e7ee1761b72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:14.924632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7851d366-97a7-41af-8d2b-7e7ee1761b72 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:14.936687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-7851d366-97a7-41af-8d2b-7e7ee1761b72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:14.936687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:14.936687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:14.936687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:14.936687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:14.936687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:14.936687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:14.936687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:14.936687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:14.936687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:14.936687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:14.936687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:14.936687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:43:14.936687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:43:14.936687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:14.938545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:14.939426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:43:14.939426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:43:14.939426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:43:14.939426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:43:14.939426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:43:14.939426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:43:14.939426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:14.939426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:14.942533 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072aa57-7cfd-4cb9-b4b8-e37f9c52d8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:14.943406 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 471/945] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:43:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:14.943886 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072aa57-7cfd-4cb9-b4b8-e37f9c52d8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:14.944580 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7851d366-97a7-41af-8d2b-7e7ee1761b72 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:14.945032 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072aa57-7cfd-4cb9-b4b8-e37f9c52d8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:14.945619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072aa57-7cfd-4cb9-b4b8-e37f9c52d8d1 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:14.946570 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072aa57-7cfd-4cb9-b4b8-e37f9c52d8d1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:14.947088 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072aa57-7cfd-4cb9-b4b8-e37f9c52d8d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:14.950169 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072aa57-7cfd-4cb9-b4b8-e37f9c52d8d1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:14.957522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 475/946] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 7 07:43:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:14.963881 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072aa57-7cfd-4cb9-b4b8-e37f9c52d8d1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:14.989663 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:15.060638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7851d366-97a7-41af-8d2b-7e7ee1761b72 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:15.073273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072aa57-7cfd-4cb9-b4b8-e37f9c52d8d1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:15.099805 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8f8ab-a87b-4811-9b3f-1cf2753366e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:15.100525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8f8ab-a87b-4811-9b3f-1cf2753366e7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:15.100770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8f8ab-a87b-4811-9b3f-1cf2753366e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:15.101084 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8f8ab-a87b-4811-9b3f-1cf2753366e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:15.101770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8f8ab-a87b-4811-9b3f-1cf2753366e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:15.102311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8f8ab-a87b-4811-9b3f-1cf2753366e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:15.103510 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8f8ab-a87b-4811-9b3f-1cf2753366e7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:15.108816 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:15.114927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96053f12-9816-427c-a7d1-5023d2067c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:15.115825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96053f12-9816-427c-a7d1-5023d2067c70 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:15.116374 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96053f12-9816-427c-a7d1-5023d2067c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:15.116880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96053f12-9816-427c-a7d1-5023d2067c70 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:15.118618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96053f12-9816-427c-a7d1-5023d2067c70 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:15.119133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96053f12-9816-427c-a7d1-5023d2067c70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:15.120732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96053f12-9816-427c-a7d1-5023d2067c70 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:15.121601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 472/947] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:43:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:15.122427 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8f8ab-a87b-4811-9b3f-1cf2753366e7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:15.132757 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 476/948] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 7 07:43:14 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:15.239162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:15.246480 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8f8ab-a87b-4811-9b3f-1cf2753366e7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:15.282989 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e72e0-241f-4f8d-831c-d773b1c8baef None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:15.283319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e72e0-241f-4f8d-831c-d773b1c8baef None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:15.283544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e72e0-241f-4f8d-831c-d773b1c8baef None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:15.284963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 477/949] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:43:15.289306 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3ee65-dd99-48ca-9996-e7b8f8ff7c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:15.289985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3ee65-dd99-48ca-9996-e7b8f8ff7c07 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:15.290264 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3ee65-dd99-48ca-9996-e7b8f8ff7c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:15.290539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3ee65-dd99-48ca-9996-e7b8f8ff7c07 None admin] PATH_INFO: `/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/617f6479b90945d3b5a63325839afb15/roles/35552dee8978468c86e59379516db233` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:15.290961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454540e-f51a-4005-8e2e-a71601710c75 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:15.291244 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454540e-f51a-4005-8e2e-a71601710c75 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:15.291521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454540e-f51a-4005-8e2e-a71601710c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:15.292943 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3ee65-dd99-48ca-9996-e7b8f8ff7c07 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a949b959e584474aafb4a05ebbabcc4, user_id=617f6479b90945d3b5a63325839afb15, role_id=35552dee8978468c86e59379516db233)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:15.294525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3ee65-dd99-48ca-9996-e7b8f8ff7c07 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:15.310619 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 473/950] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:43:15 2022] PUT /identity/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/617f6479b90945d3b5a63325839afb15/roles/35552dee8978468c86e59379516db233 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:43:15.312774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6454540e-f51a-4005-8e2e-a71601710c75 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:15.314276 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6454540e-f51a-4005-8e2e-a71601710c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:15.401183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 478/951] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:15.682960 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3ee65-dd99-48ca-9996-e7b8f8ff7c07 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:15.789628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3ee65-dd99-48ca-9996-e7b8f8ff7c07 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:15.830041 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5437c1e-44f9-416c-88bb-728acca1791e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:15.830669 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5437c1e-44f9-416c-88bb-728acca1791e None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:15.830977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5437c1e-44f9-416c-88bb-728acca1791e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:15.831255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5437c1e-44f9-416c-88bb-728acca1791e None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:15.831921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5437c1e-44f9-416c-88bb-728acca1791e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:15.833454 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5437c1e-44f9-416c-88bb-728acca1791e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:15.836165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: WARNING keystone.server.flask.common [None req-b5437c1e-44f9-416c-88bb-728acca1791e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 07 07:43:15.857560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.sql.core [None req-b5437c1e-44f9-416c-88bb-728acca1791e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 07:43:15.857560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 07 07:43:15.857560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [parameters: {'id': '43a15c1e6f92447486a18b4b82df9124', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 07 07:43:15.857560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81476) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 07:43:15.876685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-b5437c1e-44f9-416c-88bb-728acca1791e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 43a15c1e6f92447486a18b4b82df9124.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 43a15c1e6f92447486a18b4b82df9124. Sep 07 07:43:15.876685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:15.876685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 07:43:15.876685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 07:43:15.876685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 07:43:15.876685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 07:43:15.876685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 07:43:15.876685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 07:43:15.876685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 07:43:15.876685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application conn.query(q) Sep 07 07:43:15.876685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 07:43:15.876685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 07:43:15.876685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application result.read() Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 07 07:43:15.877546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application next(self.gen) Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application yield resource Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application next(self.gen) Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise self.value Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application session.commit() Sep 07 07:43:15.878494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self.session.flush() Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application compat.raise_( Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 07:43:15.879375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 07:43:15.882057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 07:43:15.882057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 07:43:15.882057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 07:43:15.882057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 07:43:15.882057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 07 07:43:15.882057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 07 07:43:15.882057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 07:43:15.882057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 07:43:15.882057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 07:43:15.882057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 07:43:15.882057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 07:43:15.882057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 07:43:15.882057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 07:43:15.882057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 07:43:15.882057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application conn.query(q) Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application result.read() Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 07:43:15.884157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 07:43:15.886088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 07:43:15.886088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 07:43:15.886088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 07:43:15.886088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 07:43:15.886088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 07:43:15.886088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 07:43:15.886088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 07:43:15.886088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 07 07:43:15.886088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [parameters: {'id': '43a15c1e6f92447486a18b4b82df9124', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 07 07:43:15.886088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 07:43:15.886088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:15.886088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:15.886088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:15.886088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:15.886088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:15.888121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:15.898392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:15.898392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 07 07:43:15.898392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 07 07:43:15.898392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:15.898392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:15.898392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 07 07:43:15.898392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 07 07:43:15.898392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 43a15c1e6f92447486a18b4b82df9124. Sep 07 07:43:15.898392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:15.898392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 474/952] 10.222.0.26 () {60 vars in 1140 bytes} [Wed Sep 7 07:43:15 2022] POST /identity/v3/projects => generated 262 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:15.899722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454540e-f51a-4005-8e2e-a71601710c75 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:16.005917 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454540e-f51a-4005-8e2e-a71601710c75 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:16.034065 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:16.043631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782ff67-b701-481e-9b92-a70a88bea860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:16.044579 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782ff67-b701-481e-9b92-a70a88bea860 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:16.045094 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782ff67-b701-481e-9b92-a70a88bea860 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:16.045589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782ff67-b701-481e-9b92-a70a88bea860 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:16.053238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782ff67-b701-481e-9b92-a70a88bea860 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:16.054921 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782ff67-b701-481e-9b92-a70a88bea860 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:16.067137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-a782ff67-b701-481e-9b92-a70a88bea860 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:16.067137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:16.067137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:16.067137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:16.067137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:16.067137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:16.067137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:16.067137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:16.067137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:16.067137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:16.067137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:16.067137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:16.067137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:43:16.067137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:43:16.067137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:16.068108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:16.069060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:43:16.069060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:43:16.069060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:43:16.069060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:43:16.069060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:43:16.069060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:43:16.069060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:16.069060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:16.071704 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 479/953] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:43:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:16.081937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5437c1e-44f9-416c-88bb-728acca1791e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:16.097918 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:16.208037 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5437c1e-44f9-416c-88bb-728acca1791e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:16.243848 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044b128-e288-4bcd-918a-352aa06b4357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:16.244490 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044b128-e288-4bcd-918a-352aa06b4357 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:16.244769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044b128-e288-4bcd-918a-352aa06b4357 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:16.245043 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044b128-e288-4bcd-918a-352aa06b4357 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:16.245712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044b128-e288-4bcd-918a-352aa06b4357 None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:16.246017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044b128-e288-4bcd-918a-352aa06b4357 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:16.247680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044b128-e288-4bcd-918a-352aa06b4357 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:16.270740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 475/954] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 7 07:43:16 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:16.278812 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268905f-8281-43bd-959c-9af51acd9333 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:16.279798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268905f-8281-43bd-959c-9af51acd9333 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:16.280322 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268905f-8281-43bd-959c-9af51acd9333 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:16.282455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 480/955] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:16.290390 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6d6d0-1226-4f78-8754-2f8870d708e4 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:16.290734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6d6d0-1226-4f78-8754-2f8870d708e4 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:16.291019 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6d6d0-1226-4f78-8754-2f8870d708e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:16.314336 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-6fe6d6d0-1226-4f78-8754-2f8870d708e4 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:16.315932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6fe6d6d0-1226-4f78-8754-2f8870d708e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:16.388978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:16.395345 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 476/956] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:16.674805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268905f-8281-43bd-959c-9af51acd9333 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:16.781540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268905f-8281-43bd-959c-9af51acd9333 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:16.817191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5262d-6f63-49b6-9e58-58b0b3d11fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:16.818064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5262d-6f63-49b6-9e58-58b0b3d11fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:16.818516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5262d-6f63-49b6-9e58-58b0b3d11fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:16.818955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5262d-6f63-49b6-9e58-58b0b3d11fe4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:16.819949 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f5262d-6f63-49b6-9e58-58b0b3d11fe4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:16.821470 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f5262d-6f63-49b6-9e58-58b0b3d11fe4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f3f5262d-6f63-49b6-9e58-58b0b3d11fe4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:16.830908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:16.832216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:16.832216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:16.832216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:16.832216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:16.832216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:16.832216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:16.832216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:16.832216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:16.832216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:16.832216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:16.832216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:16.832216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:16.832216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:16.832216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:16.832216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:16.832216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:16.833079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:16.833079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:16.833079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:16.833079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:16.833079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:16.833079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:16.833079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:16.833079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:16.833079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:16.833079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:16.833079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:16.833079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:16.833079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 07:43:16.833079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:16.833834 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 481/957] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 7 07:43:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:16.840447 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6d6d0-1226-4f78-8754-2f8870d708e4 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:16.946763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6d6d0-1226-4f78-8754-2f8870d708e4 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:16.983677 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d15ca7-f2cd-415d-bd14-690d2cf538f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:16.985059 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d15ca7-f2cd-415d-bd14-690d2cf538f1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:16.985441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d15ca7-f2cd-415d-bd14-690d2cf538f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:16.985753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d15ca7-f2cd-415d-bd14-690d2cf538f1 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:16.986413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d15ca7-f2cd-415d-bd14-690d2cf538f1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:16.986749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d15ca7-f2cd-415d-bd14-690d2cf538f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:16.988178 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d15ca7-f2cd-415d-bd14-690d2cf538f1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:16.993504 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 477/958] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 7 07:43:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:16.999203 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5262d-6f63-49b6-9e58-58b0b3d11fe4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:17.053783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:17.106053 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5262d-6f63-49b6-9e58-58b0b3d11fe4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:17.142354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9af3c-64fe-4ab0-b88c-7434bfbee854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:17.142965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9af3c-64fe-4ab0-b88c-7434bfbee854 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:17.143523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9af3c-64fe-4ab0-b88c-7434bfbee854 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:17.143797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9af3c-64fe-4ab0-b88c-7434bfbee854 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:17.144454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9af3c-64fe-4ab0-b88c-7434bfbee854 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:17.144758 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9af3c-64fe-4ab0-b88c-7434bfbee854 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:17.146239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9af3c-64fe-4ab0-b88c-7434bfbee854 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:17.151410 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 482/959] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 7 07:43:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:17.163895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d15ca7-f2cd-415d-bd14-690d2cf538f1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:17.232170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:17.276470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d15ca7-f2cd-415d-bd14-690d2cf538f1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:17.318013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7671791-6ffe-4da9-be0a-2b14d03f14e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:17.318812 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7671791-6ffe-4da9-be0a-2b14d03f14e4 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:17.319252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7671791-6ffe-4da9-be0a-2b14d03f14e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:17.319948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7671791-6ffe-4da9-be0a-2b14d03f14e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:17.321224 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7671791-6ffe-4da9-be0a-2b14d03f14e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:17.323058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7671791-6ffe-4da9-be0a-2b14d03f14e4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:17.346615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 478/960] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:43:17 2022] POST /identity/v3/endpoints => generated 349 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:43:17.513953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fc76e-43a4-4fe7-a6d1-d66db4401439 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:17.514905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fc76e-43a4-4fe7-a6d1-d66db4401439 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:17.515543 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fc76e-43a4-4fe7-a6d1-d66db4401439 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:17.517249 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 483/961] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:17.523217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d298f1-35ee-4d60-bef3-244700d33661 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:17.523511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d298f1-35ee-4d60-bef3-244700d33661 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:17.523782 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d298f1-35ee-4d60-bef3-244700d33661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:17.543888 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f3d298f1-35ee-4d60-bef3-244700d33661 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:17.545570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f3d298f1-35ee-4d60-bef3-244700d33661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:17.635820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 479/962] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:17.692510 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963467db-5ec8-4dd7-8138-81632482b397 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:17.692823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963467db-5ec8-4dd7-8138-81632482b397 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:17.693067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963467db-5ec8-4dd7-8138-81632482b397 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:17.694936 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 484/963] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:17.701487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca15bf-f2ce-4a6b-9dcb-191bdc72a69c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:17.701775 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca15bf-f2ce-4a6b-9dcb-191bdc72a69c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:17.702050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca15bf-f2ce-4a6b-9dcb-191bdc72a69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:17.721400 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-beca15bf-f2ce-4a6b-9dcb-191bdc72a69c None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:17.722594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-beca15bf-f2ce-4a6b-9dcb-191bdc72a69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:17.767548 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:17.799843 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 480/964] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:17.912600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963467db-5ec8-4dd7-8138-81632482b397 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:18.020599 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963467db-5ec8-4dd7-8138-81632482b397 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:18.054211 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:18.057046 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61d912-91d4-4dac-82d5-d3095da9f715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:18.057577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61d912-91d4-4dac-82d5-d3095da9f715 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:18.057861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61d912-91d4-4dac-82d5-d3095da9f715 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:18.058130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61d912-91d4-4dac-82d5-d3095da9f715 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:18.059239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61d912-91d4-4dac-82d5-d3095da9f715 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:18.060456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61d912-91d4-4dac-82d5-d3095da9f715 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:18.080881 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 485/965] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 7 07:43:17 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:43:18.092480 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca15bf-f2ce-4a6b-9dcb-191bdc72a69c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:18.209141 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca15bf-f2ce-4a6b-9dcb-191bdc72a69c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:18.255078 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a0c77-0834-48e3-8e45-3b05ad2bf952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:18.255712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a0c77-0834-48e3-8e45-3b05ad2bf952 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:18.255985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a0c77-0834-48e3-8e45-3b05ad2bf952 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:18.256285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a0c77-0834-48e3-8e45-3b05ad2bf952 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:18.267536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a0c77-0834-48e3-8e45-3b05ad2bf952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:43:18.272395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-1e8a0c77-0834-48e3-8e45-3b05ad2bf952 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 07:43:18.272395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:18.272395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:18.272395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:18.272395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:18.272395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:18.272395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:18.272395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:18.272395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:18.272395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:18.272395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:18.272395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:18.272395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:43:18.272395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:43:18.272395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:43:18.272395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:43:18.273320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:43:18.273320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:43:18.273320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:43:18.273320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:43:18.273320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:18.273320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:18.273320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:18.273320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:18.273320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:18.273320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:18.273320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:18.273320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:18.273320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:18.273320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:18.273320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:18.273320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:18.274675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:18.274675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:18.274675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:18.274675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:18.274675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:43:18.274675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:43:18.274675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:43:18.274675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:43:18.274675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:43:18.274675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:43:18.274675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 07:43:18.274675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:18.274675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 481/966] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Sep 7 07:43:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:18.282120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61d912-91d4-4dac-82d5-d3095da9f715 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:18.395546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61d912-91d4-4dac-82d5-d3095da9f715 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:18.430799 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68029ec7-3509-4f9a-be94-65fd85a11c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:18.431899 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68029ec7-3509-4f9a-be94-65fd85a11c7d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:18.432204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68029ec7-3509-4f9a-be94-65fd85a11c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:18.432561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68029ec7-3509-4f9a-be94-65fd85a11c7d None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:18.433215 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68029ec7-3509-4f9a-be94-65fd85a11c7d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:18.433514 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68029ec7-3509-4f9a-be94-65fd85a11c7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:18.434916 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68029ec7-3509-4f9a-be94-65fd85a11c7d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:18.443547 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 486/967] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Sep 7 07:43:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:18.449725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a0c77-0834-48e3-8e45-3b05ad2bf952 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:18.495197 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:18.572774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a0c77-0834-48e3-8e45-3b05ad2bf952 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:18.610787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c3db5-aeed-46fa-829a-b45625d879a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:18.614531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c3db5-aeed-46fa-829a-b45625d879a7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:18.614531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c3db5-aeed-46fa-829a-b45625d879a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:18.614531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c3db5-aeed-46fa-829a-b45625d879a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:18.614818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e117f6-3396-4c99-89c9-0238066cc09d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:18.614818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e117f6-3396-4c99-89c9-0238066cc09d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:18.614818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e117f6-3396-4c99-89c9-0238066cc09d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:18.614818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 487/968] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:18.619588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473c7d4-67fa-49ad-acd6-172a7cd11c49 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:18.619849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473c7d4-67fa-49ad-acd6-172a7cd11c49 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:18.626611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473c7d4-67fa-49ad-acd6-172a7cd11c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:18.626794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c3db5-aeed-46fa-829a-b45625d879a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:18.626794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c3db5-aeed-46fa-829a-b45625d879a7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:18.639537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3473c7d4-67fa-49ad-acd6-172a7cd11c49 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:18.641275 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3473c7d4-67fa-49ad-acd6-172a7cd11c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:18.645615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-5a2c3db5-aeed-46fa-829a-b45625d879a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:18.645615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:18.645615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:18.645615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:18.645615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:18.645615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:18.645615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:18.645615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:18.645615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:18.645615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:18.645615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:18.645615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:18.645615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:18.645615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:18.645615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:18.645615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:18.646740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:18.647772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:18.647772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:18.647772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:18.649480 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 482/969] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:18.655162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c3db5-aeed-46fa-829a-b45625d879a7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:18.679228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:18.723367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 488/970] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:18.760550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:18.772301 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c3db5-aeed-46fa-829a-b45625d879a7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:18.806592 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e4cd5-c3f3-4746-8d49-ea6a9fe50e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:18.807508 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e4cd5-c3f3-4746-8d49-ea6a9fe50e41 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:18.807959 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e4cd5-c3f3-4746-8d49-ea6a9fe50e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:18.808433 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e4cd5-c3f3-4746-8d49-ea6a9fe50e41 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:18.809235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e4cd5-c3f3-4746-8d49-ea6a9fe50e41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:18.809825 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e4cd5-c3f3-4746-8d49-ea6a9fe50e41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:18.811372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e4cd5-c3f3-4746-8d49-ea6a9fe50e41 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:18.824813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 483/971] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:18.837255 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473c7d4-67fa-49ad-acd6-172a7cd11c49 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:18.945124 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473c7d4-67fa-49ad-acd6-172a7cd11c49 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:18.981257 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55cfe6-71a5-48ff-8697-6951dfa9789c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:18.981887 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55cfe6-71a5-48ff-8697-6951dfa9789c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:18.982161 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55cfe6-71a5-48ff-8697-6951dfa9789c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:18.982428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55cfe6-71a5-48ff-8697-6951dfa9789c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:18.989618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55cfe6-71a5-48ff-8697-6951dfa9789c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:18.991045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55cfe6-71a5-48ff-8697-6951dfa9789c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:19.003302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ad55cfe6-71a5-48ff-8697-6951dfa9789c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 07:43:19.003302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:19.003302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:19.003302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:19.003302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:19.003302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:19.003302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:19.003302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:19.003302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:19.003302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:19.003302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:19.003302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:19.003302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:43:19.003302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:43:19.003302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:43:19.003302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:19.004325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:19.005273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:19.005273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:19.005273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:43:19.005273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:43:19.005273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:43:19.005273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:43:19.005273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:43:19.005273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:43:19.005273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 07:43:19.005273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:19.008183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 489/972] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 7 07:43:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:19.010581 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e4cd5-c3f3-4746-8d49-ea6a9fe50e41 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:19.015794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55cfe6-71a5-48ff-8697-6951dfa9789c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:19.045598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:19.123775 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e4cd5-c3f3-4746-8d49-ea6a9fe50e41 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:19.127550 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55cfe6-71a5-48ff-8697-6951dfa9789c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:19.161111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdc578-1fc6-4368-8b81-62487ad2dbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:19.162057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdc578-1fc6-4368-8b81-62487ad2dbe8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:19.162302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c564233-fc15-40ab-9c4d-0bfeae0a167b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:19.162794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdc578-1fc6-4368-8b81-62487ad2dbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:19.162966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c564233-fc15-40ab-9c4d-0bfeae0a167b None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:19.163228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c564233-fc15-40ab-9c4d-0bfeae0a167b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:19.163501 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c564233-fc15-40ab-9c4d-0bfeae0a167b None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:19.163832 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdc578-1fc6-4368-8b81-62487ad2dbe8 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:19.164167 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c564233-fc15-40ab-9c4d-0bfeae0a167b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:19.164975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdc578-1fc6-4368-8b81-62487ad2dbe8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:19.165474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdc578-1fc6-4368-8b81-62487ad2dbe8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:19.165576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c564233-fc15-40ab-9c4d-0bfeae0a167b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:19.167156 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdc578-1fc6-4368-8b81-62487ad2dbe8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:19.168109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: WARNING keystone.server.flask.common [None req-0c564233-fc15-40ab-9c4d-0bfeae0a167b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 07 07:43:19.183595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.sql.core [None req-0c564233-fc15-40ab-9c4d-0bfeae0a167b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 07:43:19.183595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 07 07:43:19.183595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [parameters: {'user_id': '0a2d77e56b2b48b291368dba9587bc87', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 07 07:43:19.183595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81475) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 07:43:19.188022 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 484/973] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Sep 7 07:43:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:19.194065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdc578-1fc6-4368-8b81-62487ad2dbe8 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:19.196884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-0c564233-fc15-40ab-9c4d-0bfeae0a167b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 07 07:43:19.196884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:19.196884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 07:43:19.196884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 07:43:19.196884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 07:43:19.196884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 07:43:19.196884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 07:43:19.196884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 07:43:19.196884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 07:43:19.196884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application conn.query(q) Sep 07 07:43:19.196884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 07:43:19.196884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 07:43:19.196884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 07:43:19.196884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result.read() Sep 07 07:43:19.196884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 07:43:19.197744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application next(self.gen) Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application yield resource Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application next(self.gen) Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise self.value Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application session.commit() Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 07:43:19.198691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.session.flush() Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application compat.raise_( Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 07:43:19.199589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 07:43:19.200522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 07:43:19.200522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 07:43:19.200522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 07 07:43:19.200522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 07 07:43:19.200522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 07:43:19.200522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 07:43:19.200522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 07:43:19.200522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 07:43:19.200522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 07:43:19.200522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 07:43:19.200522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 07:43:19.200522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 07:43:19.200522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 07:43:19.200522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 07:43:19.200522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 07:43:19.200522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception Sep 07 07:43:19.201359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 07:43:19.201359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 07:43:19.201359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 07:43:19.201359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 07:43:19.201359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 07:43:19.201359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 07:43:19.201359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 07:43:19.201359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application conn.query(q) Sep 07 07:43:19.201359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 07:43:19.201359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 07:43:19.201359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 07:43:19.201359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result.read() Sep 07 07:43:19.201359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 07:43:19.201359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 07:43:19.201359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 07:43:19.201359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 07:43:19.202176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 07:43:19.202176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 07:43:19.202176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 07:43:19.202176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 07:43:19.202176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 07:43:19.202176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 07 07:43:19.202176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [parameters: {'user_id': '0a2d77e56b2b48b291368dba9587bc87', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 07 07:43:19.202176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 07:43:19.202176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:19.202176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:19.202176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:19.202176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:19.202176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:19.202176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:19.202176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 07 07:43:19.203538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 07 07:43:19.208356 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 07 07:43:19.208356 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 07 07:43:19.208356 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 07:43:19.208356 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 07:43:19.208356 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 07 07:43:19.208356 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:19.208724 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:19.208797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 490/974] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Sep 7 07:43:19 2022] POST /identity/v3/users => generated 160 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:19.211852 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c564233-fc15-40ab-9c4d-0bfeae0a167b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:19.317190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:19.333367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c564233-fc15-40ab-9c4d-0bfeae0a167b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:19.337344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdc578-1fc6-4368-8b81-62487ad2dbe8 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:19.375345 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff2c6e-853e-4257-ab96-a16969c0c993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:19.375958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff2c6e-853e-4257-ab96-a16969c0c993 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:19.376360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff2c6e-853e-4257-ab96-a16969c0c993 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:19.376722 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff2c6e-853e-4257-ab96-a16969c0c993 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:19.377393 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0610a5-2c13-446c-82a8-66058a7d5d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:19.378215 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0610a5-2c13-446c-82a8-66058a7d5d13 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:19.378675 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0610a5-2c13-446c-82a8-66058a7d5d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:19.379114 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0610a5-2c13-446c-82a8-66058a7d5d13 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:19.389883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0610a5-2c13-446c-82a8-66058a7d5d13 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:19.393232 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0610a5-2c13-446c-82a8-66058a7d5d13 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:19.406853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ca0610a5-2c13-446c-82a8-66058a7d5d13 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 07:43:19.406853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:19.406853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:19.406853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:19.406853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:19.406853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:19.406853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:19.406853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:19.406853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:19.406853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:19.406853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:19.406853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:19.406853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:43:19.406853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:43:19.406853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:43:19.406853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:19.407806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:19.408790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:19.408790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:19.408790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:43:19.408790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:43:19.408790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:43:19.408790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:43:19.408790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:43:19.408790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:43:19.408790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 07:43:19.408790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:19.411350 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 491/975] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:43:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:19.424105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0610a5-2c13-446c-82a8-66058a7d5d13 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:19.458584 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff2c6e-853e-4257-ab96-a16969c0c993 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:19.460039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff2c6e-853e-4257-ab96-a16969c0c993 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:19.462054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:19.560468 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0610a5-2c13-446c-82a8-66058a7d5d13 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:19.598965 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 485/976] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:43:19 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 07:43:19.605348 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff2c6e-853e-4257-ab96-a16969c0c993 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:19.607317 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a51e5e-71e8-4cdd-9423-ac3be91b980d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:19.608065 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a51e5e-71e8-4cdd-9423-ac3be91b980d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:19.608446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a51e5e-71e8-4cdd-9423-ac3be91b980d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:19.608824 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a51e5e-71e8-4cdd-9423-ac3be91b980d None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:19.609668 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a51e5e-71e8-4cdd-9423-ac3be91b980d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:19.610086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a51e5e-71e8-4cdd-9423-ac3be91b980d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:19.611652 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a51e5e-71e8-4cdd-9423-ac3be91b980d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:19.660262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 492/977] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 7 07:43:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:19.720529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff2c6e-853e-4257-ab96-a16969c0c993 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:19.722254 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:19.731556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f3622-ec89-4363-aaaf-f90309aaa720 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:19.731885 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f3622-ec89-4363-aaaf-f90309aaa720 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:19.732164 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f3622-ec89-4363-aaaf-f90309aaa720 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:19.733645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 493/978] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:43:19.739810 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ae78c-f177-4271-bab3-1f6bd16d3339 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:19.740493 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ae78c-f177-4271-bab3-1f6bd16d3339 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:19.741085 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ae78c-f177-4271-bab3-1f6bd16d3339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:19.756244 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:19.762812 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40294373-01ac-4d42-a534-7f0659b87b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:19.763778 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40294373-01ac-4d42-a534-7f0659b87b3b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:19.764044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40294373-01ac-4d42-a534-7f0659b87b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:19.764347 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40294373-01ac-4d42-a534-7f0659b87b3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:19.766096 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e03ae78c-f177-4271-bab3-1f6bd16d3339 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:19.768306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e03ae78c-f177-4271-bab3-1f6bd16d3339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:19.776475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40294373-01ac-4d42-a534-7f0659b87b3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:19.777952 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40294373-01ac-4d42-a534-7f0659b87b3b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:19.794459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-40294373-01ac-4d42-a534-7f0659b87b3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:19.794459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:19.794459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:19.794459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:19.794459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:19.794459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:19.794459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:19.794459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:19.794459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:19.794459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:19.794459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:19.794459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:19.794459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:19.794459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:19.794459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:19.794459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:19.795468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:19.796750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:19.796750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:19.796750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:19.800202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 486/979] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:19.801900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40294373-01ac-4d42-a534-7f0659b87b3b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:19.861536 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 494/980] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 07:43:19.923016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40294373-01ac-4d42-a534-7f0659b87b3b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:19.960224 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8285a-6583-4384-9c3e-9e0e86649342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:19.960843 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8285a-6583-4384-9c3e-9e0e86649342 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:19.961170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8285a-6583-4384-9c3e-9e0e86649342 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:19.961433 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8285a-6583-4384-9c3e-9e0e86649342 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:19.962066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe8285a-6583-4384-9c3e-9e0e86649342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:19.962363 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe8285a-6583-4384-9c3e-9e0e86649342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:19.963819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe8285a-6583-4384-9c3e-9e0e86649342 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:19.980194 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 487/981] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:19.982419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ae78c-f177-4271-bab3-1f6bd16d3339 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:20.085062 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ae78c-f177-4271-bab3-1f6bd16d3339 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:20.118853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1b367-bbdb-47ed-ba6d-910c824167c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:20.119469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1b367-bbdb-47ed-ba6d-910c824167c3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:20.119748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1b367-bbdb-47ed-ba6d-910c824167c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:20.120020 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1b367-bbdb-47ed-ba6d-910c824167c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:20.122754 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:20.129209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1b367-bbdb-47ed-ba6d-910c824167c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:20.131175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1b367-bbdb-47ed-ba6d-910c824167c3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:20.137990 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8285a-6583-4384-9c3e-9e0e86649342 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:20.144391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d6e1b367-bbdb-47ed-ba6d-910c824167c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:20.144391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:20.144391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:20.144391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:20.144391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:20.144391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:20.144391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:20.144391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:20.144391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:20.144391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:20.144391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:20.144391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:20.144391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:43:20.144391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:43:20.144391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:20.145267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:20.150722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:43:20.150722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:43:20.150722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:43:20.150722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:43:20.150722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:43:20.150722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:43:20.150722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:20.150722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:20.150722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 495/982] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:43:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:20.152628 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1b367-bbdb-47ed-ba6d-910c824167c3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:20.154510 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:20.250901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8285a-6583-4384-9c3e-9e0e86649342 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:20.259611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1b367-bbdb-47ed-ba6d-910c824167c3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:20.289538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a809fb-4e17-4c47-ad06-20a0d4eb2cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:20.290181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a809fb-4e17-4c47-ad06-20a0d4eb2cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:20.290452 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a809fb-4e17-4c47-ad06-20a0d4eb2cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:20.290720 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a809fb-4e17-4c47-ad06-20a0d4eb2cc4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:20.291318 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a809fb-4e17-4c47-ad06-20a0d4eb2cc4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:20.292782 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a809fb-4e17-4c47-ad06-20a0d4eb2cc4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:20.297663 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f5473-c988-4155-ab4f-51634be7529c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:20.298284 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f5473-c988-4155-ab4f-51634be7529c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:20.298562 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f5473-c988-4155-ab4f-51634be7529c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:20.298839 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f5473-c988-4155-ab4f-51634be7529c None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:20.299541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f5473-c988-4155-ab4f-51634be7529c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:20.299847 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f5473-c988-4155-ab4f-51634be7529c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:20.301349 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f5473-c988-4155-ab4f-51634be7529c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-11a809fb-4e17-4c47-ad06-20a0d4eb2cc4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:20.302763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:20.303758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:20.303758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:20.303758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:20.303758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:20.303758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:20.303758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:20.303758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:20.303758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:20.303758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:20.303758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:20.303758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:20.303758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:20.303758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:20.303758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:20.303758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:20.303758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:20.304606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:20.304606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:20.304606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:20.304606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:20.304606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:20.304606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:20.304606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:20.304606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:20.304606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:20.304606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:20.304606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:20.304606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:20.304606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 07:43:20.304606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:20.309242 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 488/983] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 7 07:43:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:20.310013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a809fb-4e17-4c47-ad06-20a0d4eb2cc4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:20.323766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 496/984] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:43:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:20.325902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f5473-c988-4155-ab4f-51634be7529c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:20.388871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:20.407273 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:20.434049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f5473-c988-4155-ab4f-51634be7529c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:20.434279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a809fb-4e17-4c47-ad06-20a0d4eb2cc4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:20.468646 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3435a4-a1bf-464a-8beb-e04b1aad0af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:20.469267 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3435a4-a1bf-464a-8beb-e04b1aad0af3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:20.469544 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3435a4-a1bf-464a-8beb-e04b1aad0af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:20.469813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3435a4-a1bf-464a-8beb-e04b1aad0af3 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:20.469909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eb07c-50ff-4461-bdf2-90fdcb5bb238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:20.470432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3435a4-a1bf-464a-8beb-e04b1aad0af3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:20.470528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eb07c-50ff-4461-bdf2-90fdcb5bb238 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:20.470725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eb07c-50ff-4461-bdf2-90fdcb5bb238 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:20.470818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3435a4-a1bf-464a-8beb-e04b1aad0af3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:20.470991 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eb07c-50ff-4461-bdf2-90fdcb5bb238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:20.471695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eb07c-50ff-4461-bdf2-90fdcb5bb238 None admin] RBAC: Adding query filter params (role.id=35552dee8978468c86e59379516db233, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=617f6479b90945d3b5a63325839afb15) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:20.471994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eb07c-50ff-4461-bdf2-90fdcb5bb238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:20.472111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3435a4-a1bf-464a-8beb-e04b1aad0af3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:20.473400 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eb07c-50ff-4461-bdf2-90fdcb5bb238 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:20.476867 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 489/985] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 7 07:43:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:20.477997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacf3eb-b9b4-4a73-a4d6-06745d3d657f None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:20.478276 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacf3eb-b9b4-4a73-a4d6-06745d3d657f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:20.478542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacf3eb-b9b4-4a73-a4d6-06745d3d657f None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:20.479888 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 490/986] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:20.482524 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cacf3eb-b9b4-4a73-a4d6-06745d3d657f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:20.484166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 497/987] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:43:20 2022] GET /identity/v3/role_assignments?user.id=617f6479b90945d3b5a63325839afb15&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=35552dee8978468c86e59379516db233 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:20.485378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfd668-53bf-4882-9be2-b0a1b4bbc1f3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:20.485665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfd668-53bf-4882-9be2-b0a1b4bbc1f3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:20.485940 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfd668-53bf-4882-9be2-b0a1b4bbc1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:20.506187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d9cfd668-53bf-4882-9be2-b0a1b4bbc1f3 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:20.507698 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d9cfd668-53bf-4882-9be2-b0a1b4bbc1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:20.590673 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 498/988] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:20.607443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cacf3eb-b9b4-4a73-a4d6-06745d3d657f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:20.647084 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648eed9-635a-4dc2-96d9-ac743b9acaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:20.648156 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648eed9-635a-4dc2-96d9-ac743b9acaaf None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:20.648663 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648eed9-635a-4dc2-96d9-ac743b9acaaf None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:20.649083 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648eed9-635a-4dc2-96d9-ac743b9acaaf None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:20.649857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648eed9-635a-4dc2-96d9-ac743b9acaaf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:20.650318 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648eed9-635a-4dc2-96d9-ac743b9acaaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:20.651901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648eed9-635a-4dc2-96d9-ac743b9acaaf None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:20.660355 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 491/989] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 7 07:43:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:20.667318 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfd668-53bf-4882-9be2-b0a1b4bbc1f3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:20.721493 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:20.772819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfd668-53bf-4882-9be2-b0a1b4bbc1f3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:20.806555 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877180a-5c44-44c6-998b-e74f27719b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:20.807148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877180a-5c44-44c6-998b-e74f27719b55 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:20.807422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877180a-5c44-44c6-998b-e74f27719b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:20.807689 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877180a-5c44-44c6-998b-e74f27719b55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:20.808408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877180a-5c44-44c6-998b-e74f27719b55 None admin] RBAC: Adding query filter params (interface=admin, service_id=225f0761e1054002be68243760751f9f, region_id=RegionOne) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:20.808714 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877180a-5c44-44c6-998b-e74f27719b55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:20.810077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877180a-5c44-44c6-998b-e74f27719b55 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:20.815317 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 499/990] 10.222.0.26 () {56 vars in 1249 bytes} [Wed Sep 7 07:43:20 2022] GET /identity/v3/endpoints?service_id=225f0761e1054002be68243760751f9f&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:20.821425 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648eed9-635a-4dc2-96d9-ac743b9acaaf None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:20.831631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:20.857800 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877180a-5c44-44c6-998b-e74f27719b55 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:20.930778 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648eed9-635a-4dc2-96d9-ac743b9acaaf None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:20.968463 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf2c6fa-1dc4-4a8b-a028-d43472ae12c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:20.969177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2c6fa-1dc4-4a8b-a028-d43472ae12c1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:20.969462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2c6fa-1dc4-4a8b-a028-d43472ae12c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:20.969740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2c6fa-1dc4-4a8b-a028-d43472ae12c1 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:20.970330 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf2c6fa-1dc4-4a8b-a028-d43472ae12c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:20.971965 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf2c6fa-1dc4-4a8b-a028-d43472ae12c1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:20.974786 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877180a-5c44-44c6-998b-e74f27719b55 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:20.978572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 492/991] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 7 07:43:20 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:43:20.991400 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:21.017665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99e209-24b8-4d7a-8f34-671762ee1bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:21.018388 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99e209-24b8-4d7a-8f34-671762ee1bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:21.018660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99e209-24b8-4d7a-8f34-671762ee1bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:21.018924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99e209-24b8-4d7a-8f34-671762ee1bf5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:21.019562 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99e209-24b8-4d7a-8f34-671762ee1bf5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:21.020954 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99e209-24b8-4d7a-8f34-671762ee1bf5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-4c99e209-24b8-4d7a-8f34-671762ee1bf5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:21.031302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:21.032795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:21.032795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:21.032795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:21.032795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:21.032795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:21.032795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:21.032795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:21.032795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:21.032795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:21.032795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:21.032795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:21.032795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:21.032795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:21.032795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:21.032795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:21.032795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:21.034183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:21.034183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:21.034183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:21.034183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:21.034183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:21.034183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:21.034183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:21.034183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:21.034183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:21.034183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:21.034183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:21.034183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:21.034183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 07:43:21.034183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:21.036525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 500/992] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:43:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:21.038533 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf2c6fa-1dc4-4a8b-a028-d43472ae12c1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:21.117692 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:21.158450 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf2c6fa-1dc4-4a8b-a028-d43472ae12c1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:21.195097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63ee4f-12ee-48db-9532-eec8fcffc489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:21.195695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ee4f-12ee-48db-9532-eec8fcffc489 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:21.195965 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ee4f-12ee-48db-9532-eec8fcffc489 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:21.196258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ee4f-12ee-48db-9532-eec8fcffc489 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:21.196870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63ee4f-12ee-48db-9532-eec8fcffc489 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:21.197164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63ee4f-12ee-48db-9532-eec8fcffc489 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:21.198515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63ee4f-12ee-48db-9532-eec8fcffc489 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:21.204122 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 493/993] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 7 07:43:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:21.209406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99e209-24b8-4d7a-8f34-671762ee1bf5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:21.310504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99e209-24b8-4d7a-8f34-671762ee1bf5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:21.344946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce141aa-e6c7-4639-94ab-b407897805d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:21.345541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce141aa-e6c7-4639-94ab-b407897805d7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:21.345813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce141aa-e6c7-4639-94ab-b407897805d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:21.346078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce141aa-e6c7-4639-94ab-b407897805d7 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:21.346799 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce141aa-e6c7-4639-94ab-b407897805d7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:21.347099 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce141aa-e6c7-4639-94ab-b407897805d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:21.348530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce141aa-e6c7-4639-94ab-b407897805d7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:21.358110 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 501/994] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 7 07:43:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:21.360299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63ee4f-12ee-48db-9532-eec8fcffc489 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:21.473292 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63ee4f-12ee-48db-9532-eec8fcffc489 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:21.506260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d599f1-ae2a-4205-9569-cb647ac4259a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:21.506859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d599f1-ae2a-4205-9569-cb647ac4259a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:21.507133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d599f1-ae2a-4205-9569-cb647ac4259a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:21.507396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d599f1-ae2a-4205-9569-cb647ac4259a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:21.508074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d599f1-ae2a-4205-9569-cb647ac4259a None admin] RBAC: Adding query filter params (interface=public, service_id=8524381f38e14f139c969c2e539f41a9, region_id=RegionOne) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:21.508401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d599f1-ae2a-4205-9569-cb647ac4259a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:21.509760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d599f1-ae2a-4205-9569-cb647ac4259a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:21.511264 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:21.520125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 494/995] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Sep 7 07:43:21 2022] GET /identity/v3/endpoints?service_id=8524381f38e14f139c969c2e539f41a9&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:21.523275 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce141aa-e6c7-4639-94ab-b407897805d7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:21.569252 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:21.626651 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce141aa-e6c7-4639-94ab-b407897805d7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:21.659924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464b424-b316-4e02-b431-f8fe263f53ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:21.660553 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464b424-b316-4e02-b431-f8fe263f53ac None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:21.660828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464b424-b316-4e02-b431-f8fe263f53ac None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:21.661095 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464b424-b316-4e02-b431-f8fe263f53ac None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:21.661682 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464b424-b316-4e02-b431-f8fe263f53ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:21.663023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464b424-b316-4e02-b431-f8fe263f53ac None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:21.676510 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 502/996] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 7 07:43:21 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:43:22.703330 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224e277-dd27-40de-b30b-915a021ce40d None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:22.703816 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224e277-dd27-40de-b30b-915a021ce40d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:22.703816 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224e277-dd27-40de-b30b-915a021ce40d None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:22.705303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 495/997] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:22.710306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e49220-b919-4c0b-8586-353c594a6858 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:22.710681 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e49220-b919-4c0b-8586-353c594a6858 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:22.710883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e49220-b919-4c0b-8586-353c594a6858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:22.729368 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-44e49220-b919-4c0b-8586-353c594a6858 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:22.730555 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-44e49220-b919-4c0b-8586-353c594a6858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:22.799347 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 503/998] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:22 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:23.067469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224e277-dd27-40de-b30b-915a021ce40d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:23.152055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8a8f3-0f0e-4028-b743-8a6a2814dcfe None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:23.152431 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8a8f3-0f0e-4028-b743-8a6a2814dcfe None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:23.152638 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8a8f3-0f0e-4028-b743-8a6a2814dcfe None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:23.153983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 504/999] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:23.159488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348bfa9-3b62-4257-b45a-ec10b4d33eaa None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:23.159773 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348bfa9-3b62-4257-b45a-ec10b4d33eaa None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:23.160047 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348bfa9-3b62-4257-b45a-ec10b4d33eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:23.173631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224e277-dd27-40de-b30b-915a021ce40d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:23.179591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b348bfa9-3b62-4257-b45a-ec10b4d33eaa None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:23.181209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b348bfa9-3b62-4257-b45a-ec10b4d33eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:23.209261 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85f931-3711-4e86-b852-f5878d12d7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:23.210214 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85f931-3711-4e86-b852-f5878d12d7ea None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:23.210498 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85f931-3711-4e86-b852-f5878d12d7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:23.210771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85f931-3711-4e86-b852-f5878d12d7ea None admin] PATH_INFO: `/v3/services/image` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:23.211411 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85f931-3711-4e86-b852-f5878d12d7ea None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:23.212805 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85f931-3711-4e86-b852-f5878d12d7ea None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:23.213971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-2a85f931-3711-4e86-b852-f5878d12d7ea None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:23.222138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:23.223135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:23.223135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:23.223135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:23.223135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:23.223135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:23.223135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:23.223135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:23.223135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:23.223135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:23.223135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:23.223135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:23.223135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:23.223135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:23.223135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:23.223135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:23.223135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:23.223961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:23.223961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:23.223961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:23.223961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:23.223961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:23.223961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:23.223961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:23.223961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:23.223961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:23.223961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:23.223961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:23.223961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:23.223961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 07:43:23.223961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:23.223961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 496/1000] 10.222.0.26 () {56 vars in 1105 bytes} [Wed Sep 7 07:43:23 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:23.229035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85f931-3711-4e86-b852-f5878d12d7ea None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:23.260804 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 505/1001] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:23 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 07:43:23.294778 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:23.335487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85f931-3711-4e86-b852-f5878d12d7ea None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:23.370507 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a93cc-6414-4fdb-a4e5-7a74a0ddc1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:23.371108 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a93cc-6414-4fdb-a4e5-7a74a0ddc1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:23.371386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a93cc-6414-4fdb-a4e5-7a74a0ddc1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:23.371656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a93cc-6414-4fdb-a4e5-7a74a0ddc1c5 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:23.372296 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a93cc-6414-4fdb-a4e5-7a74a0ddc1c5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:23.372596 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a93cc-6414-4fdb-a4e5-7a74a0ddc1c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:23.373960 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a93cc-6414-4fdb-a4e5-7a74a0ddc1c5 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:23.383182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 497/1002] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:23.388042 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348bfa9-3b62-4257-b45a-ec10b4d33eaa None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:23.497089 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348bfa9-3b62-4257-b45a-ec10b4d33eaa None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:23.532207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ecee2-7a4d-4cf6-84cc-c344e61a2269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:23.532810 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ecee2-7a4d-4cf6-84cc-c344e61a2269 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:23.533089 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ecee2-7a4d-4cf6-84cc-c344e61a2269 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:23.533362 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ecee2-7a4d-4cf6-84cc-c344e61a2269 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:23.533985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ecee2-7a4d-4cf6-84cc-c344e61a2269 None admin] RBAC: Adding query filter params (type=image) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:23.534286 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ecee2-7a4d-4cf6-84cc-c344e61a2269 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:23.535639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ecee2-7a4d-4cf6-84cc-c344e61a2269 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:23.538018 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a93cc-6414-4fdb-a4e5-7a74a0ddc1c5 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:23.540595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 506/1003] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:23 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:23.652394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a93cc-6414-4fdb-a4e5-7a74a0ddc1c5 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:23.680705 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:23.691343 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30aa94-48b2-425d-b095-64e14745b24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:23.691983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30aa94-48b2-425d-b095-64e14745b24a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:23.692285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30aa94-48b2-425d-b095-64e14745b24a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:23.692555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30aa94-48b2-425d-b095-64e14745b24a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:23.693135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30aa94-48b2-425d-b095-64e14745b24a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:23.694775 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30aa94-48b2-425d-b095-64e14745b24a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-ad30aa94-48b2-425d-b095-64e14745b24a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:23.704810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:23.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:23.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:23.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:23.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:23.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:23.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:23.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:23.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:23.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:23.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:23.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:23.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:23.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:23.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:23.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:23.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:23.707133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:23.707133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:23.707133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:23.707133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:23.707133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:23.707133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:23.707133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:23.707133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:23.707133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:23.707133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:23.707133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:23.707133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:23.707133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 07:43:23.707133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:23.712107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 498/1004] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 7 07:43:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:23.712271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ecee2-7a4d-4cf6-84cc-c344e61a2269 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:23.754375 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:23.827287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ecee2-7a4d-4cf6-84cc-c344e61a2269 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:23.859439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92bdd3-6310-4b10-8fe2-5f4ee3c1a144 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:23.859913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92bdd3-6310-4b10-8fe2-5f4ee3c1a144 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:23.860322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92bdd3-6310-4b10-8fe2-5f4ee3c1a144 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:23.861732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 499/1005] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:43:23.862465 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b7d76-3188-47e8-ae95-c25d0a1f912b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:23.863068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b7d76-3188-47e8-ae95-c25d0a1f912b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:23.863415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b7d76-3188-47e8-ae95-c25d0a1f912b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:23.863605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b7d76-3188-47e8-ae95-c25d0a1f912b None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:23.864258 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b7d76-3188-47e8-ae95-c25d0a1f912b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:23.864558 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b7d76-3188-47e8-ae95-c25d0a1f912b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:23.865903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b7d76-3188-47e8-ae95-c25d0a1f912b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:23.868331 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc64c0-9615-4d3a-8d61-4d68eee7452f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:23.868610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc64c0-9615-4d3a-8d61-4d68eee7452f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:23.868878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc64c0-9615-4d3a-8d61-4d68eee7452f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:23.871355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 507/1006] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 7 07:43:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:23.877436 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b7d76-3188-47e8-ae95-c25d0a1f912b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:23.887025 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-06dc64c0-9615-4d3a-8d61-4d68eee7452f None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:23.888230 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-06dc64c0-9615-4d3a-8d61-4d68eee7452f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:23.967049 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 500/1007] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:23 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 07:43:23.980505 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:23.987987 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b7d76-3188-47e8-ae95-c25d0a1f912b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:24.023281 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6e57a-049b-471d-9774-33b5fc61845e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:24.023878 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6e57a-049b-471d-9774-33b5fc61845e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:24.024179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6e57a-049b-471d-9774-33b5fc61845e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:24.024451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6e57a-049b-471d-9774-33b5fc61845e None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:24.025068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6e57a-049b-471d-9774-33b5fc61845e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:24.026964 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6e57a-049b-471d-9774-33b5fc61845e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:24.026964 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6e57a-049b-471d-9774-33b5fc61845e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:24.032051 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 508/1008] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 7 07:43:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:24.037484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc64c0-9615-4d3a-8d61-4d68eee7452f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:24.138799 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc64c0-9615-4d3a-8d61-4d68eee7452f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:24.159095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:24.174650 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d5630-a299-43c6-b357-576aa74c3c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:24.175251 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d5630-a299-43c6-b357-576aa74c3c9c None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:24.175526 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d5630-a299-43c6-b357-576aa74c3c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:24.175794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d5630-a299-43c6-b357-576aa74c3c9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:24.176485 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d5630-a299-43c6-b357-576aa74c3c9c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:24.177913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d5630-a299-43c6-b357-576aa74c3c9c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:24.198919 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 501/1009] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:43:24 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:43:24.244212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6e57a-049b-471d-9774-33b5fc61845e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:24.356617 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6e57a-049b-471d-9774-33b5fc61845e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:24.405472 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700aa99-31d2-4b21-8cb6-ff4aa59eca0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:24.406808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700aa99-31d2-4b21-8cb6-ff4aa59eca0d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:24.407052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700aa99-31d2-4b21-8cb6-ff4aa59eca0d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:24.407317 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700aa99-31d2-4b21-8cb6-ff4aa59eca0d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:24.407880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e700aa99-31d2-4b21-8cb6-ff4aa59eca0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:24.409256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e700aa99-31d2-4b21-8cb6-ff4aa59eca0d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-e700aa99-31d2-4b21-8cb6-ff4aa59eca0d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:24.421152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:24.422459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:24.422459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:24.422459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:24.422459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:24.422459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:24.422459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:24.422459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:24.422459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:24.422459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:24.422459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:24.422459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:24.422459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:24.422459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:24.422459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:24.422459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:24.422459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:24.423929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:24.423929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:24.423929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:24.423929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:24.423929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:24.423929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:24.423929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:24.423929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:24.423929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:24.423929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:24.423929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:24.423929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:24.423929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 07:43:24.423929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:24.423929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 509/1010] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:43:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:24.428272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d5630-a299-43c6-b357-576aa74c3c9c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:24.536961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d5630-a299-43c6-b357-576aa74c3c9c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:24.571358 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890927b-8c83-4af5-a0e6-593ee2473b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:24.571965 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890927b-8c83-4af5-a0e6-593ee2473b3b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:24.572325 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890927b-8c83-4af5-a0e6-593ee2473b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:24.572595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890927b-8c83-4af5-a0e6-593ee2473b3b None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:24.573215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890927b-8c83-4af5-a0e6-593ee2473b3b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:24.573515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890927b-8c83-4af5-a0e6-593ee2473b3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:24.574876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890927b-8c83-4af5-a0e6-593ee2473b3b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:24.585091 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 502/1011] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 7 07:43:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:24.586467 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700aa99-31d2-4b21-8cb6-ff4aa59eca0d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:24.607352 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:24.697263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700aa99-31d2-4b21-8cb6-ff4aa59eca0d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:24.730227 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f77b4-c6e5-4aa7-8dc4-d0b9e15208cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:24.730831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f77b4-c6e5-4aa7-8dc4-d0b9e15208cd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:24.731083 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f77b4-c6e5-4aa7-8dc4-d0b9e15208cd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:24.731349 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f77b4-c6e5-4aa7-8dc4-d0b9e15208cd None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:24.731964 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f77b4-c6e5-4aa7-8dc4-d0b9e15208cd None admin] RBAC: Adding query filter params (type=compute) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:24.732287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f77b4-c6e5-4aa7-8dc4-d0b9e15208cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:24.734149 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f77b4-c6e5-4aa7-8dc4-d0b9e15208cd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:24.743432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 510/1012] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 7 07:43:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:24.745105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890927b-8c83-4af5-a0e6-593ee2473b3b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:24.795054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:24.852073 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890927b-8c83-4af5-a0e6-593ee2473b3b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:24.886792 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3a027-7906-434a-b464-2d30eaf1c39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:24.887390 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3a027-7906-434a-b464-2d30eaf1c39e None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:24.887663 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3a027-7906-434a-b464-2d30eaf1c39e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:24.887931 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3a027-7906-434a-b464-2d30eaf1c39e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:24.888607 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3a027-7906-434a-b464-2d30eaf1c39e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:24.890056 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3a027-7906-434a-b464-2d30eaf1c39e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:24.916106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 503/1013] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:43:24 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:43:25.735576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f00bf8-2fb6-40a4-8b6b-329a0154bffd None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:25.736107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f00bf8-2fb6-40a4-8b6b-329a0154bffd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:25.736107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f00bf8-2fb6-40a4-8b6b-329a0154bffd None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:25.737549 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 511/1014] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:25.742657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85b8b0-2b9e-49ab-bfdb-3c8715d9aff3 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:25.742957 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85b8b0-2b9e-49ab-bfdb-3c8715d9aff3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:25.743238 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85b8b0-2b9e-49ab-bfdb-3c8715d9aff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:25.761473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-db85b8b0-2b9e-49ab-bfdb-3c8715d9aff3 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:25.762659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-db85b8b0-2b9e-49ab-bfdb-3c8715d9aff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:25.838244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 504/1015] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:25 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:26.103316 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f00bf8-2fb6-40a4-8b6b-329a0154bffd None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:26.186614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:26.207219 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f00bf8-2fb6-40a4-8b6b-329a0154bffd None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:26.241720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755154b7-a7a0-4bf7-aa27-188cead41d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:26.242327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755154b7-a7a0-4bf7-aa27-188cead41d59 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:26.242602 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755154b7-a7a0-4bf7-aa27-188cead41d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:26.242877 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755154b7-a7a0-4bf7-aa27-188cead41d59 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:26.243534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755154b7-a7a0-4bf7-aa27-188cead41d59 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:26.244925 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755154b7-a7a0-4bf7-aa27-188cead41d59 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:26.264116 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 512/1016] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 7 07:43:26 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:43:26.439551 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91760161-6890-43b6-9af6-f671f5434e30 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:26.439888 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91760161-6890-43b6-9af6-f671f5434e30 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:26.440130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91760161-6890-43b6-9af6-f671f5434e30 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:26.441491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 505/1017] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:43:26.447039 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4855ff0-c17e-415c-8b13-708ba03e9ee9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:26.447328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4855ff0-c17e-415c-8b13-708ba03e9ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:26.447609 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4855ff0-c17e-415c-8b13-708ba03e9ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:26.466778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e4855ff0-c17e-415c-8b13-708ba03e9ee9 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:26.468048 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e4855ff0-c17e-415c-8b13-708ba03e9ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:26.545177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 513/1018] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:26.821798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91760161-6890-43b6-9af6-f671f5434e30 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:26.851271 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:26.931490 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91760161-6890-43b6-9af6-f671f5434e30 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:26.968433 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda6d3b-78d3-485a-bc4c-a29e5604a362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:26.969220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda6d3b-78d3-485a-bc4c-a29e5604a362 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:26.969501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda6d3b-78d3-485a-bc4c-a29e5604a362 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:26.969772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda6d3b-78d3-485a-bc4c-a29e5604a362 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:26.970893 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda6d3b-78d3-485a-bc4c-a29e5604a362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:26.972362 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda6d3b-78d3-485a-bc4c-a29e5604a362 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:26.974093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 506/1019] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:26.979700 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4855ff0-c17e-415c-8b13-708ba03e9ee9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:27.052579 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:27.092141 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4855ff0-c17e-415c-8b13-708ba03e9ee9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:27.130800 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a5922-6653-4b89-a550-173848e45d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:27.131411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a5922-6653-4b89-a550-173848e45d6b None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:27.131744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a5922-6653-4b89-a550-173848e45d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:27.132023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a5922-6653-4b89-a550-173848e45d6b None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:27.132710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a5922-6653-4b89-a550-173848e45d6b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:27.134131 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a5922-6653-4b89-a550-173848e45d6b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:27.172113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 514/1020] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 7 07:43:26 2022] POST /identity/v3/projects => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:43:27.201176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9d72f-aab0-4beb-8667-5a53aa60b330 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:27.201537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9d72f-aab0-4beb-8667-5a53aa60b330 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:27.201811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9d72f-aab0-4beb-8667-5a53aa60b330 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:27.203842 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 507/1021] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:27.209685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c8e82-0e86-4569-9858-0f9a4bbc1c2f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:27.210012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c8e82-0e86-4569-9858-0f9a4bbc1c2f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:27.210350 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c8e82-0e86-4569-9858-0f9a4bbc1c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:27.230614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-166c8e82-0e86-4569-9858-0f9a4bbc1c2f None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:27.231854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-166c8e82-0e86-4569-9858-0f9a4bbc1c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:27.305269 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 515/1022] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:27.580814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9d72f-aab0-4beb-8667-5a53aa60b330 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:27.680389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:27.689226 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9d72f-aab0-4beb-8667-5a53aa60b330 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:27.724740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae5774-bcef-4281-a596-f8aa6228132a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:27.725367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae5774-bcef-4281-a596-f8aa6228132a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:27.725644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae5774-bcef-4281-a596-f8aa6228132a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:27.725915 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae5774-bcef-4281-a596-f8aa6228132a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:27.736751 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae5774-bcef-4281-a596-f8aa6228132a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:43:27.741569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-0bae5774-bcef-4281-a596-f8aa6228132a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 07:43:27.741569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:27.741569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:27.741569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:27.741569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:27.741569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:27.741569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:27.741569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:27.741569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:27.741569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:27.741569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:27.741569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:27.741569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:43:27.741569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:43:27.741569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:43:27.741569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:43:27.742473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:43:27.742473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:43:27.742473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:43:27.742473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:43:27.742473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:27.742473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:27.742473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:27.742473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:27.742473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:27.742473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:27.742473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:27.742473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:27.742473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:27.742473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:27.742473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:27.742473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:27.743322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:27.743322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:27.743322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:27.743322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:27.743322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:43:27.743322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:43:27.743322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:43:27.743322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:43:27.743322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:43:27.743322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:43:27.743322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 07:43:27.743322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:27.750818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 508/1023] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Sep 7 07:43:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:27.750963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c8e82-0e86-4569-9858-0f9a4bbc1c2f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:27.800847 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:27.853917 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c8e82-0e86-4569-9858-0f9a4bbc1c2f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:27.888869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb1624-68c2-4b6f-945c-abc6a34b094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:27.889488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb1624-68c2-4b6f-945c-abc6a34b094b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:27.889808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb1624-68c2-4b6f-945c-abc6a34b094b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:27.890295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb1624-68c2-4b6f-945c-abc6a34b094b None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:27.891067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb1624-68c2-4b6f-945c-abc6a34b094b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:27.891373 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb1624-68c2-4b6f-945c-abc6a34b094b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:27.892764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb1624-68c2-4b6f-945c-abc6a34b094b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:27.906696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 516/1024] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Sep 7 07:43:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:27.908431 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae5774-bcef-4281-a596-f8aa6228132a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:28.017047 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae5774-bcef-4281-a596-f8aa6228132a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:28.052176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da749e1-deb9-4593-a041-abcb02e1662f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:28.052858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da749e1-deb9-4593-a041-abcb02e1662f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:28.053143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da749e1-deb9-4593-a041-abcb02e1662f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:28.053417 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da749e1-deb9-4593-a041-abcb02e1662f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:28.064566 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da749e1-deb9-4593-a041-abcb02e1662f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:28.066026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da749e1-deb9-4593-a041-abcb02e1662f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:28.081389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-2da749e1-deb9-4593-a041-abcb02e1662f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:28.081389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:28.081389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:28.081389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:28.081389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:28.081389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:28.081389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:28.081389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:28.081389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:28.081389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:28.081389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:28.081389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:28.081389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:28.081389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:28.081389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:28.081389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:28.082432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:28.083303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:28.083303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:28.083303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:28.083303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 509/1025] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:28.089636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb1624-68c2-4b6f-945c-abc6a34b094b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:28.195929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb1624-68c2-4b6f-945c-abc6a34b094b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:28.209060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:28.240355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909d12a-038d-4f72-8482-599bf4b8641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:28.240959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909d12a-038d-4f72-8482-599bf4b8641f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:28.241234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909d12a-038d-4f72-8482-599bf4b8641f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:28.241499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909d12a-038d-4f72-8482-599bf4b8641f None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:28.242130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909d12a-038d-4f72-8482-599bf4b8641f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:28.242428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909d12a-038d-4f72-8482-599bf4b8641f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:28.243774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909d12a-038d-4f72-8482-599bf4b8641f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:28.255740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 517/1026] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:28.261147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da749e1-deb9-4593-a041-abcb02e1662f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:28.272293 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:28.367985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da749e1-deb9-4593-a041-abcb02e1662f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:28.406581 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586200f-d4d3-47fb-b903-d9d3eb385197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:28.407203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586200f-d4d3-47fb-b903-d9d3eb385197 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:28.407477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586200f-d4d3-47fb-b903-d9d3eb385197 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:28.407744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586200f-d4d3-47fb-b903-d9d3eb385197 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:28.418942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586200f-d4d3-47fb-b903-d9d3eb385197 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:28.418942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586200f-d4d3-47fb-b903-d9d3eb385197 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:28.430913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-f586200f-d4d3-47fb-b903-d9d3eb385197 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:43:28.430913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:28.430913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:28.430913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:28.430913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:28.430913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:28.430913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:28.430913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:28.430913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:28.430913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:28.430913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:28.430913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:28.430913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:43:28.430913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:43:28.430913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:43:28.430913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:28.432036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:28.433217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:28.433217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:28.433217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:43:28.433217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:43:28.433217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:43:28.433217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:43:28.433217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:43:28.433217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:43:28.433217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:43:28.433217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:28.435499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 510/1027] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 7 07:43:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:28.442902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909d12a-038d-4f72-8482-599bf4b8641f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:28.463941 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb461090-d5cc-4ce6-96e9-808532bdabc9 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:28.464270 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb461090-d5cc-4ce6-96e9-808532bdabc9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:28.464515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb461090-d5cc-4ce6-96e9-808532bdabc9 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:28.465979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 511/1028] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:43:28.471151 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a543f1-a667-4e76-ab20-1d33a0cdae81 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:28.471919 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a543f1-a667-4e76-ab20-1d33a0cdae81 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:28.472413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a543f1-a667-4e76-ab20-1d33a0cdae81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:28.495388 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a5a543f1-a667-4e76-ab20-1d33a0cdae81 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:28.496610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a5a543f1-a667-4e76-ab20-1d33a0cdae81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:28.550575 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909d12a-038d-4f72-8482-599bf4b8641f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:28.565352 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:28.573795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 512/1029] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 07:43:28.587116 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8170f-ec52-4b26-a385-0c5e46d7bc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:28.587718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8170f-ec52-4b26-a385-0c5e46d7bc5a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:28.587996 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8170f-ec52-4b26-a385-0c5e46d7bc5a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:28.588294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8170f-ec52-4b26-a385-0c5e46d7bc5a None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:28.589018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8170f-ec52-4b26-a385-0c5e46d7bc5a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:28.589323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8170f-ec52-4b26-a385-0c5e46d7bc5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:28.590735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8170f-ec52-4b26-a385-0c5e46d7bc5a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:28.609338 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 518/1030] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 7 07:43:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:28.615130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a543f1-a667-4e76-ab20-1d33a0cdae81 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:28.718649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a543f1-a667-4e76-ab20-1d33a0cdae81 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:28.753509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ae7cd-1a1b-4e0c-845f-bbabcb381845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:28.754111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae7cd-1a1b-4e0c-845f-bbabcb381845 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:28.754389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae7cd-1a1b-4e0c-845f-bbabcb381845 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:28.754658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae7cd-1a1b-4e0c-845f-bbabcb381845 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:28.816898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:28.824750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ae7cd-1a1b-4e0c-845f-bbabcb381845 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:28.826206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ae7cd-1a1b-4e0c-845f-bbabcb381845 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:28.840289 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8170f-ec52-4b26-a385-0c5e46d7bc5a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:28.871791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:28.933592 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 513/1031] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:43:28 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:43:28.939303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ae7cd-1a1b-4e0c-845f-bbabcb381845 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:28.945844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8170f-ec52-4b26-a385-0c5e46d7bc5a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:28.980626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cf8d6-6d03-4c9c-9ec7-3f2f1a2b0d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:28.981251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cf8d6-6d03-4c9c-9ec7-3f2f1a2b0d50 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:28.981536 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cf8d6-6d03-4c9c-9ec7-3f2f1a2b0d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:28.981806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cf8d6-6d03-4c9c-9ec7-3f2f1a2b0d50 None admin] PATH_INFO: `/v3/services/image` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:28.982386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cf8d6-6d03-4c9c-9ec7-3f2f1a2b0d50 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:28.984382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cf8d6-6d03-4c9c-9ec7-3f2f1a2b0d50 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-2f8cf8d6-6d03-4c9c-9ec7-3f2f1a2b0d50 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:28.993439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:28.994417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:28.994417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:28.994417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:28.994417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:28.994417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:28.994417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:28.994417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:28.994417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:28.994417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:28.994417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:28.994417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:28.994417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:28.994417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:28.994417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:28.994417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:28.994417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:28.995273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:28.995273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:28.995273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:28.995273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:28.995273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:28.995273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:28.995273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:28.995273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:28.995273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:28.995273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:28.995273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:28.995273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:28.995273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 07:43:28.995273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:28.995273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 519/1032] 10.222.0.26 () {56 vars in 1105 bytes} [Wed Sep 7 07:43:28 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:29.000827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cf8d6-6d03-4c9c-9ec7-3f2f1a2b0d50 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:29.041469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:29.048850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ae7cd-1a1b-4e0c-845f-bbabcb381845 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:29.088325 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2edd83-763d-4ed9-9f42-8756f90ddd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:29.088871 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2edd83-763d-4ed9-9f42-8756f90ddd15 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:29.089149 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2edd83-763d-4ed9-9f42-8756f90ddd15 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:29.089422 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2edd83-763d-4ed9-9f42-8756f90ddd15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:29.096385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:29.102242 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2edd83-763d-4ed9-9f42-8756f90ddd15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:29.103688 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2edd83-763d-4ed9-9f42-8756f90ddd15 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:29.104711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cf8d6-6d03-4c9c-9ec7-3f2f1a2b0d50 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:29.120351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-ad2edd83-763d-4ed9-9f42-8756f90ddd15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:29.120351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:29.120351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:29.120351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:29.120351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:29.120351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:29.120351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:29.120351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:29.120351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:29.120351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:29.120351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:29.120351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:29.120351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:29.120351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:29.120351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:29.120351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:29.121315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:29.122204 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:29.122204 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:29.122204 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:29.122204 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 514/1033] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:29.127052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2edd83-763d-4ed9-9f42-8756f90ddd15 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:29.141809 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e9509-f7e9-4741-9341-d4c2c209bb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:29.142419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e9509-f7e9-4741-9341-d4c2c209bb1d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:29.142701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e9509-f7e9-4741-9341-d4c2c209bb1d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:29.142974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e9509-f7e9-4741-9341-d4c2c209bb1d None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:29.143606 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e9509-f7e9-4741-9341-d4c2c209bb1d None admin] RBAC: Adding query filter params (name=image) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:29.143901 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e9509-f7e9-4741-9341-d4c2c209bb1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:29.145295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e9509-f7e9-4741-9341-d4c2c209bb1d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:29.150444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 520/1034] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:29.156438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e9509-f7e9-4741-9341-d4c2c209bb1d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:29.236457 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2edd83-763d-4ed9-9f42-8756f90ddd15 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:29.266067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e9509-f7e9-4741-9341-d4c2c209bb1d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:29.275804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e50459-ac43-47e9-a3b9-6443c0f236df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:29.276434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e50459-ac43-47e9-a3b9-6443c0f236df None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:29.276712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e50459-ac43-47e9-a3b9-6443c0f236df None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:29.276980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e50459-ac43-47e9-a3b9-6443c0f236df None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:29.277645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e50459-ac43-47e9-a3b9-6443c0f236df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:29.277953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e50459-ac43-47e9-a3b9-6443c0f236df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:29.279309 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e50459-ac43-47e9-a3b9-6443c0f236df None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:29.292170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 515/1035] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:29.298136 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e50459-ac43-47e9-a3b9-6443c0f236df None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:29.310238 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:29.314207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334f0ec-8825-4141-ba91-d3207fbf2f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:29.314832 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334f0ec-8825-4141-ba91-d3207fbf2f22 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:29.315102 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334f0ec-8825-4141-ba91-d3207fbf2f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:29.315366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334f0ec-8825-4141-ba91-d3207fbf2f22 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:29.316001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334f0ec-8825-4141-ba91-d3207fbf2f22 None admin] RBAC: Adding query filter params (type=image) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:29.316455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334f0ec-8825-4141-ba91-d3207fbf2f22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:29.317874 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334f0ec-8825-4141-ba91-d3207fbf2f22 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:29.329471 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 521/1036] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:29 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:29.330625 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334f0ec-8825-4141-ba91-d3207fbf2f22 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:29.355875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:29.412519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e50459-ac43-47e9-a3b9-6443c0f236df None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:29.441288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334f0ec-8825-4141-ba91-d3207fbf2f22 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:29.447574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7b394-c1d9-43e5-8d16-e00647e1bbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:29.448227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7b394-c1d9-43e5-8d16-e00647e1bbd3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:29.448502 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7b394-c1d9-43e5-8d16-e00647e1bbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:29.448778 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7b394-c1d9-43e5-8d16-e00647e1bbd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:29.456350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7b394-c1d9-43e5-8d16-e00647e1bbd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:29.457786 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7b394-c1d9-43e5-8d16-e00647e1bbd3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:29.469058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-3ff7b394-c1d9-43e5-8d16-e00647e1bbd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:29.469058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:29.469058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:29.469058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:29.469058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:29.469058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:29.469058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:29.469058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:29.469058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:29.469058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:29.469058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:29.469058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:29.469058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:43:29.469058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:43:29.469058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:29.469969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:29.470859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:43:29.470859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:43:29.470859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:43:29.470859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:43:29.470859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:43:29.470859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:43:29.470859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:29.470859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:29.470859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 516/1037] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:43:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:29.471344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647088c8-77a5-4f32-8961-6a862730e450 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:29.471553 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647088c8-77a5-4f32-8961-6a862730e450 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:29.471844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647088c8-77a5-4f32-8961-6a862730e450 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:29.473061 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 517/1038] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:29.475718 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647088c8-77a5-4f32-8961-6a862730e450 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:29.485257 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262fed81-4027-4c11-8e9c-eccfc1518d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:29.485943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262fed81-4027-4c11-8e9c-eccfc1518d8b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:29.486219 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262fed81-4027-4c11-8e9c-eccfc1518d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:29.486487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262fed81-4027-4c11-8e9c-eccfc1518d8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:29.487542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262fed81-4027-4c11-8e9c-eccfc1518d8b None admin] RBAC: Adding query filter params (interface=public, service_id=a20d979464cc4c5cba65570215568b80, region_id=RegionOne) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:29.487852 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262fed81-4027-4c11-8e9c-eccfc1518d8b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:29.489545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262fed81-4027-4c11-8e9c-eccfc1518d8b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:29.494450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 522/1039] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Sep 7 07:43:29 2022] GET /identity/v3/endpoints?service_id=a20d979464cc4c5cba65570215568b80&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:29.495483 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c5959-73a6-4fdb-afdf-a0e7bd624bd9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:29.495760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c5959-73a6-4fdb-afdf-a0e7bd624bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:29.496030 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c5959-73a6-4fdb-afdf-a0e7bd624bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:29.513940 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6e5c5959-73a6-4fdb-afdf-a0e7bd624bd9 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:29.515733 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6e5c5959-73a6-4fdb-afdf-a0e7bd624bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:29.549460 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:29.587604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 523/1040] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:29.588751 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c5959-73a6-4fdb-afdf-a0e7bd624bd9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:29.606058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:29.609534 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647088c8-77a5-4f32-8961-6a862730e450 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:29.646572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b340334-0355-4d6e-b72f-6e55d57df569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:29.647268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b340334-0355-4d6e-b72f-6e55d57df569 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:29.647621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b340334-0355-4d6e-b72f-6e55d57df569 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:29.647971 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b340334-0355-4d6e-b72f-6e55d57df569 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:29.648817 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b340334-0355-4d6e-b72f-6e55d57df569 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:29.649200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b340334-0355-4d6e-b72f-6e55d57df569 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:29.650764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b340334-0355-4d6e-b72f-6e55d57df569 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:29.662960 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 518/1041] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:43:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:29.668666 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b340334-0355-4d6e-b72f-6e55d57df569 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:29.697020 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c5959-73a6-4fdb-afdf-a0e7bd624bd9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:29.736214 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581295ca-0458-486c-bbe3-732ed917aa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:29.737029 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581295ca-0458-486c-bbe3-732ed917aa56 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:29.737305 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581295ca-0458-486c-bbe3-732ed917aa56 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:29.737568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581295ca-0458-486c-bbe3-732ed917aa56 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:29.738161 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581295ca-0458-486c-bbe3-732ed917aa56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:29.739495 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581295ca-0458-486c-bbe3-732ed917aa56 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:29.760464 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 524/1042] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 7 07:43:29 2022] GET /identity/v3/services => generated 1957 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:43:29.774818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b340334-0355-4d6e-b72f-6e55d57df569 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:29.804922 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:29.815216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bcc81-02de-4d09-b271-33fa4132f682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:29.815928 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bcc81-02de-4d09-b271-33fa4132f682 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:29.816313 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bcc81-02de-4d09-b271-33fa4132f682 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:29.816678 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bcc81-02de-4d09-b271-33fa4132f682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:29.817476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8bcc81-02de-4d09-b271-33fa4132f682 None admin] RBAC: Adding query filter params (role.id=35552dee8978468c86e59379516db233, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=81d5689e9a344bafb74ca423ffd2b42a) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:29.817857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8bcc81-02de-4d09-b271-33fa4132f682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:29.819651 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8bcc81-02de-4d09-b271-33fa4132f682 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:29.832121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 519/1043] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:43:29 2022] GET /identity/v3/role_assignments?user.id=81d5689e9a344bafb74ca423ffd2b42a&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=35552dee8978468c86e59379516db233 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:29.860031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581295ca-0458-486c-bbe3-732ed917aa56 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:29.956803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:29.980280 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581295ca-0458-486c-bbe3-732ed917aa56 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:30.018756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89e4c6-f44a-4cfe-a876-2402ec263d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:30.019390 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89e4c6-f44a-4cfe-a876-2402ec263d20 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:30.019667 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89e4c6-f44a-4cfe-a876-2402ec263d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:30.019936 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89e4c6-f44a-4cfe-a876-2402ec263d20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:30.021066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89e4c6-f44a-4cfe-a876-2402ec263d20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:30.022487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89e4c6-f44a-4cfe-a876-2402ec263d20 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:30.024300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 525/1044] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:30.029715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bcc81-02de-4d09-b271-33fa4132f682 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:30.135382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bcc81-02de-4d09-b271-33fa4132f682 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:30.170176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e62eb8b-13b0-4b5c-af90-60f204c945a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:30.170782 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62eb8b-13b0-4b5c-af90-60f204c945a9 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:30.171125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62eb8b-13b0-4b5c-af90-60f204c945a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:30.171396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62eb8b-13b0-4b5c-af90-60f204c945a9 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:30.172050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e62eb8b-13b0-4b5c-af90-60f204c945a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:30.173494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e62eb8b-13b0-4b5c-af90-60f204c945a9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:30.196921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 520/1045] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:43:30 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:43:31.953317 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04c7e4-3a0f-4a0d-a020-a4fb45feee2e None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:31.953827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04c7e4-3a0f-4a0d-a020-a4fb45feee2e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:31.953934 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04c7e4-3a0f-4a0d-a020-a4fb45feee2e None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:31.955236 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 526/1046] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:31.960403 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fadf74-e907-446c-842a-d0f8759ab6e9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:31.960773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fadf74-e907-446c-842a-d0f8759ab6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:31.960951 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fadf74-e907-446c-842a-d0f8759ab6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:31.979961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a0fadf74-e907-446c-842a-d0f8759ab6e9 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:31.981196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a0fadf74-e907-446c-842a-d0f8759ab6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:32.013114 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba4a1b-b89a-4837-a3fd-757c17553637 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:32.013517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba4a1b-b89a-4837-a3fd-757c17553637 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:32.013685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba4a1b-b89a-4837-a3fd-757c17553637 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:32.015743 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 527/1047] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:43:32.021079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701bfa2-98b5-4b70-86b3-ff853f8dedc1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:32.021361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701bfa2-98b5-4b70-86b3-ff853f8dedc1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:32.021631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701bfa2-98b5-4b70-86b3-ff853f8dedc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:32.027750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:32.041466 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f701bfa2-98b5-4b70-86b3-ff853f8dedc1 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:32.042847 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f701bfa2-98b5-4b70-86b3-ff853f8dedc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:32.053922 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 521/1048] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:32.113297 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 528/1049] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 07:43:32.327829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fadf74-e907-446c-842a-d0f8759ab6e9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:32.387368 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701bfa2-98b5-4b70-86b3-ff853f8dedc1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:32.433353 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fadf74-e907-446c-842a-d0f8759ab6e9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:32.467820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a86a2-6de5-43d1-819e-565e03fc62af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:32.468664 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a86a2-6de5-43d1-819e-565e03fc62af None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:32.468808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:32.468940 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a86a2-6de5-43d1-819e-565e03fc62af None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:32.469207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a86a2-6de5-43d1-819e-565e03fc62af None admin] PATH_INFO: `/v3/services/image` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:32.469768 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a86a2-6de5-43d1-819e-565e03fc62af None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:32.471139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a86a2-6de5-43d1-819e-565e03fc62af None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-370a86a2-6de5-43d1-819e-565e03fc62af None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:32.479672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:32.480758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:32.480758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:32.480758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:32.480758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:32.480758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:32.480758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:32.480758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:43:32.480758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:43:32.480758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:43:32.480758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:43:32.480758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:32.480758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:32.480758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:32.480758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:32.480758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:32.480758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:32.481590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:32.481590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:32.481590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:32.481590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:32.481590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:32.481590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:32.481590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:32.481590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:32.481590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:32.481590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:32.481590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:43:32.481590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:43:32.481590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 07:43:32.481590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:32.481590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 522/1050] 10.222.0.26 () {56 vars in 1105 bytes} [Wed Sep 7 07:43:32 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:32.482556 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c45fd-c9ac-483d-bc1a-f17a4d94ded9 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:32.482556 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c45fd-c9ac-483d-bc1a-f17a4d94ded9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:32.482556 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c45fd-c9ac-483d-bc1a-f17a4d94ded9 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:32.483850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 523/1051] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:32.486528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c45fd-c9ac-483d-bc1a-f17a4d94ded9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:32.493789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701bfa2-98b5-4b70-86b3-ff853f8dedc1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:32.527713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae06581-a59e-46b1-bdab-b195a69a0339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:32.528359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae06581-a59e-46b1-bdab-b195a69a0339 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:32.528632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae06581-a59e-46b1-bdab-b195a69a0339 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:32.528896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae06581-a59e-46b1-bdab-b195a69a0339 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:32.531079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:32.539173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae06581-a59e-46b1-bdab-b195a69a0339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:43:32.543895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-3ae06581-a59e-46b1-bdab-b195a69a0339 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 07:43:32.543895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:32.543895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:32.543895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:32.543895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:32.543895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:32.543895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:32.543895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:32.543895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:32.543895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:32.543895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:32.543895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:32.543895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:43:32.543895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:43:32.543895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:43:32.543895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:43:32.544873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:43:32.544873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:43:32.544873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:43:32.544873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:43:32.544873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:32.544873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:32.544873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:32.544873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:32.544873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:32.544873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:32.544873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:32.544873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:32.544873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:32.544873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:32.544873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:32.544873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:32.545727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:32.545727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:32.545727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:32.545727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:32.545727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:43:32.545727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:43:32.545727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:43:32.545727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:43:32.545727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:43:32.545727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:43:32.545727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 07:43:32.545727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:32.545727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 529/1052] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Sep 7 07:43:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:32.546613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89fe56-511c-41bf-975c-873bda209e2e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:32.546613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89fe56-511c-41bf-975c-873bda209e2e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:32.546744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89fe56-511c-41bf-975c-873bda209e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:32.565237 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2e89fe56-511c-41bf-975c-873bda209e2e None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:32.566437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2e89fe56-511c-41bf-975c-873bda209e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:32.607168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c45fd-c9ac-483d-bc1a-f17a4d94ded9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:32.636785 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 530/1053] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:32.637618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89fe56-511c-41bf-975c-873bda209e2e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:32.651608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b5693-d2b8-48a8-a6d6-b7fe12970900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:32.652428 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b5693-d2b8-48a8-a6d6-b7fe12970900 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:32.652703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b5693-d2b8-48a8-a6d6-b7fe12970900 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:32.653087 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b5693-d2b8-48a8-a6d6-b7fe12970900 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:32.653704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b5693-d2b8-48a8-a6d6-b7fe12970900 None admin] RBAC: Adding query filter params (name=image) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:32.654008 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b5693-d2b8-48a8-a6d6-b7fe12970900 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:32.655707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b5693-d2b8-48a8-a6d6-b7fe12970900 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:32.664148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 524/1054] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:32.666662 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b5693-d2b8-48a8-a6d6-b7fe12970900 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:32.707699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:32.743750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89fe56-511c-41bf-975c-873bda209e2e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:32.778416 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986f0fa-95dc-44d3-9b40-aa1af43e60f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:32.778749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b5693-d2b8-48a8-a6d6-b7fe12970900 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:32.779015 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986f0fa-95dc-44d3-9b40-aa1af43e60f5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:32.779287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986f0fa-95dc-44d3-9b40-aa1af43e60f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:32.779551 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986f0fa-95dc-44d3-9b40-aa1af43e60f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:32.780365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986f0fa-95dc-44d3-9b40-aa1af43e60f5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:32.780668 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986f0fa-95dc-44d3-9b40-aa1af43e60f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:32.782018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986f0fa-95dc-44d3-9b40-aa1af43e60f5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:32.786806 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:32.792004 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 531/1055] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Sep 7 07:43:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:32.799576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986f0fa-95dc-44d3-9b40-aa1af43e60f5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:32.825641 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a7e57-5218-467c-afab-ebdb970fe323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:32.826245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a7e57-5218-467c-afab-ebdb970fe323 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:32.826523 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a7e57-5218-467c-afab-ebdb970fe323 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:32.826795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a7e57-5218-467c-afab-ebdb970fe323 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:32.827464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a7e57-5218-467c-afab-ebdb970fe323 None admin] RBAC: Adding query filter params (type=image) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:32.827748 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a7e57-5218-467c-afab-ebdb970fe323 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:32.829149 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a7e57-5218-467c-afab-ebdb970fe323 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:32.838655 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 525/1056] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:32 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:32.839770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a7e57-5218-467c-afab-ebdb970fe323 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:32.908432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986f0fa-95dc-44d3-9b40-aa1af43e60f5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:32.937199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:32.943290 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a7e57-5218-467c-afab-ebdb970fe323 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:32.944646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0716c3-3cbb-4325-adcf-da91b52be929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:32.945275 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0716c3-3cbb-4325-adcf-da91b52be929 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:32.945553 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0716c3-3cbb-4325-adcf-da91b52be929 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:32.945906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0716c3-3cbb-4325-adcf-da91b52be929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:32.957959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0716c3-3cbb-4325-adcf-da91b52be929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:32.959401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0716c3-3cbb-4325-adcf-da91b52be929 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:32.973591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-fa0716c3-3cbb-4325-adcf-da91b52be929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:32.973591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:32.973591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:32.973591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:32.973591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:32.973591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:32.973591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:32.973591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:32.973591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:32.973591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:32.973591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:32.973591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:32.973591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:32.973591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:32.973591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:32.973591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:32.974613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:32.975500 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:32.975500 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:32.975500 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:32.975500 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 532/1057] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:32.975500 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0716c3-3cbb-4325-adcf-da91b52be929 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:32.987217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3c702-a1dd-45b1-bb23-89a09c26df93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:32.987820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3c702-a1dd-45b1-bb23-89a09c26df93 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:32.988132 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3c702-a1dd-45b1-bb23-89a09c26df93 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:32.988401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3c702-a1dd-45b1-bb23-89a09c26df93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:32.989058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3c702-a1dd-45b1-bb23-89a09c26df93 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:32.990411 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3c702-a1dd-45b1-bb23-89a09c26df93 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:33.009761 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 526/1058] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:43:32 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:43:33.010557 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3c702-a1dd-45b1-bb23-89a09c26df93 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:33.042610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:33.089025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0716c3-3cbb-4325-adcf-da91b52be929 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:33.138959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e97304-9fa1-4631-a184-62a473f81f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:33.140439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3c702-a1dd-45b1-bb23-89a09c26df93 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:33.140616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e97304-9fa1-4631-a184-62a473f81f71 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:33.140846 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e97304-9fa1-4631-a184-62a473f81f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:33.141195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e97304-9fa1-4631-a184-62a473f81f71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:33.155359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e97304-9fa1-4631-a184-62a473f81f71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:43:33.160614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-45e97304-9fa1-4631-a184-62a473f81f71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:43:33.160614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:33.160614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:33.160614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:33.160614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:33.160614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:33.160614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:33.160614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:33.160614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:33.160614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:33.160614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:33.160614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:33.160614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:43:33.160614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:43:33.160614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:43:33.160614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:43:33.161605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:43:33.161605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:43:33.161605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:43:33.161605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:43:33.161605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:33.161605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:33.161605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:33.161605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:33.161605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:33.161605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:33.161605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:33.161605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:33.161605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:33.161605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:33.161605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:33.161605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:33.162439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:33.162439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:33.162439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:33.162439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:33.162439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:43:33.162439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:43:33.162439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:43:33.162439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:43:33.162439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:43:33.162439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:43:33.162439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:43:33.162439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:33.162439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 533/1059] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:43:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:33.167817 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e97304-9fa1-4631-a184-62a473f81f71 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:33.197423 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd779d5-3bad-4a3d-a56e-6b4d6efce9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:33.197423 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd779d5-3bad-4a3d-a56e-6b4d6efce9cc None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:33.197777 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd779d5-3bad-4a3d-a56e-6b4d6efce9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:33.198083 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd779d5-3bad-4a3d-a56e-6b4d6efce9cc None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:33.199261 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd779d5-3bad-4a3d-a56e-6b4d6efce9cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:33.199370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd779d5-3bad-4a3d-a56e-6b4d6efce9cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:33.201693 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd779d5-3bad-4a3d-a56e-6b4d6efce9cc None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:33.214350 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:33.218308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 527/1060] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:33.223969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd779d5-3bad-4a3d-a56e-6b4d6efce9cc None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:33.287559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e97304-9fa1-4631-a184-62a473f81f71 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:33.310936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:33.324492 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261be1c-2edb-4342-896e-f55d23b6ee34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:33.325092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261be1c-2edb-4342-896e-f55d23b6ee34 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:33.325366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261be1c-2edb-4342-896e-f55d23b6ee34 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:33.325635 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261be1c-2edb-4342-896e-f55d23b6ee34 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:33.326308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261be1c-2edb-4342-896e-f55d23b6ee34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:33.326589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261be1c-2edb-4342-896e-f55d23b6ee34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:33.328350 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261be1c-2edb-4342-896e-f55d23b6ee34 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:33.336369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd779d5-3bad-4a3d-a56e-6b4d6efce9cc None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:33.336723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 534/1061] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:43:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:33.342357 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261be1c-2edb-4342-896e-f55d23b6ee34 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:33.380736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a9c3d-1993-4f77-8a33-ee507105c578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:33.380736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a9c3d-1993-4f77-8a33-ee507105c578 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:33.380736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a9c3d-1993-4f77-8a33-ee507105c578 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:33.380736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a9c3d-1993-4f77-8a33-ee507105c578 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:33.388525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a9c3d-1993-4f77-8a33-ee507105c578 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:33.390052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a9c3d-1993-4f77-8a33-ee507105c578 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:33.401974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-083a9c3d-1993-4f77-8a33-ee507105c578 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:43:33.401974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:33.401974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:33.401974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:33.401974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:33.401974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:33.401974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:33.401974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:33.401974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:33.401974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:33.401974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:33.401974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:33.401974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:43:33.401974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:43:33.401974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:43:33.401974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:33.402923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:33.403884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:33.403884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:33.403884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:43:33.403884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:43:33.403884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:43:33.403884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:43:33.403884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:43:33.403884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:43:33.403884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:43:33.403884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:33.403884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 528/1062] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 7 07:43:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:33.409081 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a9c3d-1993-4f77-8a33-ee507105c578 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:33.456102 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:33.459729 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261be1c-2edb-4342-896e-f55d23b6ee34 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:33.498825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d65857-75af-4444-aaaf-eab530d104b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:33.499763 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d65857-75af-4444-aaaf-eab530d104b8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:33.500369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d65857-75af-4444-aaaf-eab530d104b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:33.500640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d65857-75af-4444-aaaf-eab530d104b8 None admin] PATH_INFO: `/v3/users/6441d8ee1198410395397093e472f5b8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:33.515168 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d65857-75af-4444-aaaf-eab530d104b8 None admin] RBAC: Authorizing `identity:get_user(user_id=6441d8ee1198410395397093e472f5b8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:33.517282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d65857-75af-4444-aaaf-eab530d104b8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:33.523654 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 535/1063] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:43:33 2022] GET /identity/v3/users/6441d8ee1198410395397093e472f5b8 => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:33.523819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a9c3d-1993-4f77-8a33-ee507105c578 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:33.524831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d65857-75af-4444-aaaf-eab530d104b8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:33.559646 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd5fe5-e054-420f-88d1-760446b5dbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:33.560334 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd5fe5-e054-420f-88d1-760446b5dbd3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:33.560610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd5fe5-e054-420f-88d1-760446b5dbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:33.560881 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd5fe5-e054-420f-88d1-760446b5dbd3 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:33.561724 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd5fe5-e054-420f-88d1-760446b5dbd3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:33.562050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd5fe5-e054-420f-88d1-760446b5dbd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:33.563482 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd5fe5-e054-420f-88d1-760446b5dbd3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:33.578850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:33.588813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 529/1064] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 7 07:43:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:33.589683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd5fe5-e054-420f-88d1-760446b5dbd3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:33.639390 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d65857-75af-4444-aaaf-eab530d104b8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:33.677461 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ee621-67d1-471d-a01c-19dabc87dccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:33.678065 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ee621-67d1-471d-a01c-19dabc87dccc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:33.678345 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ee621-67d1-471d-a01c-19dabc87dccc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:33.678620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ee621-67d1-471d-a01c-19dabc87dccc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:33.695453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd5fe5-e054-420f-88d1-760446b5dbd3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:33.711496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:33.732740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab06e9c-4862-4f23-91bc-417a3bda66c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:33.733905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab06e9c-4862-4f23-91bc-417a3bda66c6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:33.734333 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab06e9c-4862-4f23-91bc-417a3bda66c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:33.734778 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab06e9c-4862-4f23-91bc-417a3bda66c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:33.745363 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab06e9c-4862-4f23-91bc-417a3bda66c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:33.746821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab06e9c-4862-4f23-91bc-417a3bda66c6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:33.756767 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ee621-67d1-471d-a01c-19dabc87dccc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:33.758613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ee621-67d1-471d-a01c-19dabc87dccc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:33.764029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-1ab06e9c-4862-4f23-91bc-417a3bda66c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:33.764029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:33.764029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:33.764029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:33.764029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:33.764029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:33.764029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:33.764029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:33.764029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:33.764029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:33.764029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:33.764029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:33.764029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:33.764029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:33.764029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:33.764029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:33.765516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:33.766637 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:33.766637 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:33.766637 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:33.768042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 530/1065] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:33.773634 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab06e9c-4862-4f23-91bc-417a3bda66c6 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:33.832249 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:33.873522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 536/1066] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:43:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 07:43:33.875544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ee621-67d1-471d-a01c-19dabc87dccc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:33.892340 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab06e9c-4862-4f23-91bc-417a3bda66c6 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:33.931012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bb625-72b6-4778-9223-6fbee60999a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:33.931977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bb625-72b6-4778-9223-6fbee60999a9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:33.932428 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bb625-72b6-4778-9223-6fbee60999a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:33.932697 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bb625-72b6-4778-9223-6fbee60999a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:33.933348 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037bb625-72b6-4778-9223-6fbee60999a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:33.933645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037bb625-72b6-4778-9223-6fbee60999a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:33.935034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037bb625-72b6-4778-9223-6fbee60999a9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:33.943745 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:33.952148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 531/1067] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:33.957423 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bb625-72b6-4778-9223-6fbee60999a9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:33.993904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ee621-67d1-471d-a01c-19dabc87dccc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:34.029647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1561b4-0fe1-42bc-85a7-cdef2a6cdb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:34.030885 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1561b4-0fe1-42bc-85a7-cdef2a6cdb20 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:34.031353 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1561b4-0fe1-42bc-85a7-cdef2a6cdb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:34.031769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1561b4-0fe1-42bc-85a7-cdef2a6cdb20 None admin] PATH_INFO: `/v3/projects/7cfe99a9523d430381740dd9a483d06f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:34.032930 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1561b4-0fe1-42bc-85a7-cdef2a6cdb20 None admin] RBAC: Authorizing `identity:get_project(project_id=7cfe99a9523d430381740dd9a483d06f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:34.034360 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1561b4-0fe1-42bc-85a7-cdef2a6cdb20 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:34.036266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 537/1068] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:43:33 2022] GET /identity/v3/projects/7cfe99a9523d430381740dd9a483d06f => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:34.041846 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1561b4-0fe1-42bc-85a7-cdef2a6cdb20 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:34.067672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bb625-72b6-4778-9223-6fbee60999a9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:34.082365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:34.104284 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7319bc-aaa3-4e31-9b31-aaae9608cfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:34.104908 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7319bc-aaa3-4e31-9b31-aaae9608cfe9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:34.105430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7319bc-aaa3-4e31-9b31-aaae9608cfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:34.105430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7319bc-aaa3-4e31-9b31-aaae9608cfe9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:34.112438 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7319bc-aaa3-4e31-9b31-aaae9608cfe9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:34.113859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7319bc-aaa3-4e31-9b31-aaae9608cfe9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:34.125568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-7f7319bc-aaa3-4e31-9b31-aaae9608cfe9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:34.125568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:34.125568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:34.125568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:34.125568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:34.125568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:34.125568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:34.125568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:34.125568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:34.125568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:34.125568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:34.125568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:34.125568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:43:34.125568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:43:34.125568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:34.126436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:34.127326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:43:34.127326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:43:34.127326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:43:34.127326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:43:34.127326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:43:34.127326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:43:34.127326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:34.127326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:34.131535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 532/1069] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:43:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:34.132384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7319bc-aaa3-4e31-9b31-aaae9608cfe9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:34.153633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1561b4-0fe1-42bc-85a7-cdef2a6cdb20 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:34.184231 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:34.195066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5a085-8d45-49de-8654-97e48317191f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:34.195824 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5a085-8d45-49de-8654-97e48317191f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:34.196122 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5a085-8d45-49de-8654-97e48317191f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:34.196708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5a085-8d45-49de-8654-97e48317191f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:34.197713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5a085-8d45-49de-8654-97e48317191f None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=7cfe99a9523d430381740dd9a483d06f, user.id=6441d8ee1198410395397093e472f5b8) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:34.198022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5a085-8d45-49de-8654-97e48317191f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:34.199600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5a085-8d45-49de-8654-97e48317191f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:34.207217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 538/1070] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:43:34 2022] GET /identity/v3/role_assignments?user.id=6441d8ee1198410395397093e472f5b8&scope.project.id=7cfe99a9523d430381740dd9a483d06f&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:34.242230 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7319bc-aaa3-4e31-9b31-aaae9608cfe9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:34.282465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557e058-3ff4-4719-b15f-8c836b616bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:34.283010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557e058-3ff4-4719-b15f-8c836b616bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:34.283291 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557e058-3ff4-4719-b15f-8c836b616bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:34.283566 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557e058-3ff4-4719-b15f-8c836b616bf6 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:34.284306 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6557e058-3ff4-4719-b15f-8c836b616bf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:34.284618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6557e058-3ff4-4719-b15f-8c836b616bf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:34.286136 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6557e058-3ff4-4719-b15f-8c836b616bf6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:34.305282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 533/1071] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:43:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:34.307171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5a085-8d45-49de-8654-97e48317191f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:34.422684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5a085-8d45-49de-8654-97e48317191f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:34.457990 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999418ed-18d6-4cb7-8cb2-a8c26df9c7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:34.458643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999418ed-18d6-4cb7-8cb2-a8c26df9c7e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:34.458920 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999418ed-18d6-4cb7-8cb2-a8c26df9c7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:34.459190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999418ed-18d6-4cb7-8cb2-a8c26df9c7e3 None admin] PATH_INFO: `/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/81d5689e9a344bafb74ca423ffd2b42a/roles/35552dee8978468c86e59379516db233` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:34.461377 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999418ed-18d6-4cb7-8cb2-a8c26df9c7e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a949b959e584474aafb4a05ebbabcc4, user_id=81d5689e9a344bafb74ca423ffd2b42a, role_id=35552dee8978468c86e59379516db233)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:34.462789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999418ed-18d6-4cb7-8cb2-a8c26df9c7e3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:34.484117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 539/1072] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:43:34 2022] PUT /identity/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/81d5689e9a344bafb74ca423ffd2b42a/roles/35552dee8978468c86e59379516db233 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:43:35.231763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa106b-5abc-433b-ac13-83c0822fb546 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:35.232406 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa106b-5abc-433b-ac13-83c0822fb546 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:35.232406 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa106b-5abc-433b-ac13-83c0822fb546 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:35.234359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 534/1073] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:35.239433 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9c84e-6590-42f3-81d9-e5fb4e9ecbdc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:35.239827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9c84e-6590-42f3-81d9-e5fb4e9ecbdc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:35.240006 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9c84e-6590-42f3-81d9-e5fb4e9ecbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:35.258347 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ccf9c84e-6590-42f3-81d9-e5fb4e9ecbdc None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:35.259535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ccf9c84e-6590-42f3-81d9-e5fb4e9ecbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:35.306450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:35.329454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 540/1074] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:35.599559 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa106b-5abc-433b-ac13-83c0822fb546 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:35.631488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:35.705190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa106b-5abc-433b-ac13-83c0822fb546 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:35.740483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0006b36-0476-4d4f-bc95-96c7c9798af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:35.741084 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0006b36-0476-4d4f-bc95-96c7c9798af1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:35.741362 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0006b36-0476-4d4f-bc95-96c7c9798af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:35.741634 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0006b36-0476-4d4f-bc95-96c7c9798af1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:35.810705 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0006b36-0476-4d4f-bc95-96c7c9798af1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:35.812439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0006b36-0476-4d4f-bc95-96c7c9798af1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:35.842386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:35.913528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 535/1075] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:43:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:43:35.919143 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9c84e-6590-42f3-81d9-e5fb4e9ecbdc None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:36.018943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9c84e-6590-42f3-81d9-e5fb4e9ecbdc None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:36.053755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b147082-d921-4789-afbf-1bc8233cb83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:36.054432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b147082-d921-4789-afbf-1bc8233cb83f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:36.054712 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b147082-d921-4789-afbf-1bc8233cb83f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:36.054983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b147082-d921-4789-afbf-1bc8233cb83f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:36.065668 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b147082-d921-4789-afbf-1bc8233cb83f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:36.067095 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b147082-d921-4789-afbf-1bc8233cb83f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:36.081941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-0b147082-d921-4789-afbf-1bc8233cb83f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:36.081941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:36.081941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:36.081941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:36.081941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:36.081941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:36.081941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:36.081941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:36.081941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:36.081941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:36.081941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:36.081941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:36.081941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:36.081941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:36.081941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:36.081941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:36.082960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:36.084424 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:36.084424 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:36.084424 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:36.088122 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 541/1076] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:36.089161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0006b36-0476-4d4f-bc95-96c7c9798af1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:36.190540 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0006b36-0476-4d4f-bc95-96c7c9798af1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:36.221546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:36.227713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829fe4f0-ad11-4368-a6e3-46d31b4f879d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:36.228588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829fe4f0-ad11-4368-a6e3-46d31b4f879d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:36.229026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829fe4f0-ad11-4368-a6e3-46d31b4f879d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:36.229471 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829fe4f0-ad11-4368-a6e3-46d31b4f879d None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:36.230247 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829fe4f0-ad11-4368-a6e3-46d31b4f879d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:36.230704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829fe4f0-ad11-4368-a6e3-46d31b4f879d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:36.232246 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829fe4f0-ad11-4368-a6e3-46d31b4f879d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:36.252110 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 536/1077] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:36.411614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10581b28-faca-439a-9e34-03e1dac95eb9 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:36.412302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10581b28-faca-439a-9e34-03e1dac95eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:36.412302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10581b28-faca-439a-9e34-03e1dac95eb9 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:36.414882 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 542/1078] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:36.419182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96780afb-6c8d-491f-91f6-cd0abd37373c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:36.419479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96780afb-6c8d-491f-91f6-cd0abd37373c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:36.419757 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96780afb-6c8d-491f-91f6-cd0abd37373c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:36.439081 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-96780afb-6c8d-491f-91f6-cd0abd37373c None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:36.440295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-96780afb-6c8d-491f-91f6-cd0abd37373c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:36.512186 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 537/1079] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:36.642875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f7121-2120-4e91-a826-da79309e0e5a None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:36.645023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f7121-2120-4e91-a826-da79309e0e5a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:36.645023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f7121-2120-4e91-a826-da79309e0e5a None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:36.645023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 543/1080] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:36.650023 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c17344-8a92-462f-a172-3a3d9b8c60b3 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:36.650308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c17344-8a92-462f-a172-3a3d9b8c60b3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:36.650579 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c17344-8a92-462f-a172-3a3d9b8c60b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:36.668720 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e5c17344-8a92-462f-a172-3a3d9b8c60b3 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:36.669882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e5c17344-8a92-462f-a172-3a3d9b8c60b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:36.737617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 538/1081] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:36.786931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f7121-2120-4e91-a826-da79309e0e5a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:36.819172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:36.889039 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f7121-2120-4e91-a826-da79309e0e5a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:36.922033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41443f3e-310a-4b24-899c-b3809446bb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:36.922651 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41443f3e-310a-4b24-899c-b3809446bb37 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:36.922932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41443f3e-310a-4b24-899c-b3809446bb37 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:36.923207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41443f3e-310a-4b24-899c-b3809446bb37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:36.932996 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41443f3e-310a-4b24-899c-b3809446bb37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:43:36.938687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-41443f3e-310a-4b24-899c-b3809446bb37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:43:36.938687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:36.938687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:36.938687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:36.938687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:36.938687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:36.938687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:36.938687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:36.938687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:36.938687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:36.938687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:36.938687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:36.938687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:43:36.938687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:43:36.938687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:43:36.938687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:43:36.939783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:43:36.939783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:43:36.939783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:43:36.939783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:43:36.939783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:36.939783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:36.939783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:36.939783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:36.939783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:36.939783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:36.939783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:36.939783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:36.939783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:36.939783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:36.939783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:36.939783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:36.940708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:36.940708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:36.940708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:36.940708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:36.940708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:43:36.940708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:43:36.940708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:43:36.940708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:43:36.940708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:43:36.940708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:43:36.940708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:43:36.940708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:36.940708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 544/1082] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:43:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:36.946355 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c17344-8a92-462f-a172-3a3d9b8c60b3 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:36.997735 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:37.003584 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41443f3e-310a-4b24-899c-b3809446bb37 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:37.045399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c17344-8a92-462f-a172-3a3d9b8c60b3 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:37.079107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bac78-5f99-4796-a0a8-5aff30f66672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:37.079710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bac78-5f99-4796-a0a8-5aff30f66672 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:37.079987 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bac78-5f99-4796-a0a8-5aff30f66672 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:37.080285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bac78-5f99-4796-a0a8-5aff30f66672 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:37.080937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bac78-5f99-4796-a0a8-5aff30f66672 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:37.081236 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bac78-5f99-4796-a0a8-5aff30f66672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:37.082611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bac78-5f99-4796-a0a8-5aff30f66672 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:37.090915 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 539/1083] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:43:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:37.098881 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bac78-5f99-4796-a0a8-5aff30f66672 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:37.103784 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:37.113928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41443f3e-310a-4b24-899c-b3809446bb37 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:37.147816 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e43278-fb06-444e-94ba-87caca03a5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:37.148468 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e43278-fb06-444e-94ba-87caca03a5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:37.148740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e43278-fb06-444e-94ba-87caca03a5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:37.149013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e43278-fb06-444e-94ba-87caca03a5a1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:37.159473 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e43278-fb06-444e-94ba-87caca03a5a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:43:37.164240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-43e43278-fb06-444e-94ba-87caca03a5a1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 07:43:37.164240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:37.164240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:37.164240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:37.164240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:37.164240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:37.164240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:37.164240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:37.164240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:37.164240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:37.164240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:37.164240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:37.164240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:43:37.164240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:43:37.164240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:43:37.164240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:43:37.165333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:43:37.165333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:43:37.165333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:43:37.165333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:43:37.165333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:37.165333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:37.165333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:37.165333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:37.165333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:37.165333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:37.165333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:37.165333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:37.165333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:37.165333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:37.165333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:37.165333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:37.166601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:37.166601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:37.166601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:37.166601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:37.166601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:43:37.166601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:43:37.166601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:43:37.166601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:43:37.166601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:43:37.166601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:43:37.166601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 07:43:37.166601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:37.166601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 545/1084] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Sep 7 07:43:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:37.171390 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e43278-fb06-444e-94ba-87caca03a5a1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:37.201135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bac78-5f99-4796-a0a8-5aff30f66672 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:37.215444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:37.236248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98a5a3-438a-44d5-bb00-9c04d8ec0d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:37.236875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98a5a3-438a-44d5-bb00-9c04d8ec0d73 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:37.237131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98a5a3-438a-44d5-bb00-9c04d8ec0d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:37.237399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98a5a3-438a-44d5-bb00-9c04d8ec0d73 None admin] PATH_INFO: `/v3/users/6441d8ee1198410395397093e472f5b8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:37.238512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98a5a3-438a-44d5-bb00-9c04d8ec0d73 None admin] RBAC: Authorizing `identity:get_user(user_id=6441d8ee1198410395397093e472f5b8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:37.239937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98a5a3-438a-44d5-bb00-9c04d8ec0d73 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:37.241700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 540/1085] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:43:37 2022] GET /identity/v3/users/6441d8ee1198410395397093e472f5b8 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:37.247108 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98a5a3-438a-44d5-bb00-9c04d8ec0d73 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:37.275600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e43278-fb06-444e-94ba-87caca03a5a1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:37.309710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b5176-752d-4ea0-99ad-835c5ac65dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:37.310218 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b5176-752d-4ea0-99ad-835c5ac65dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:37.310498 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b5176-752d-4ea0-99ad-835c5ac65dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:37.310768 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b5176-752d-4ea0-99ad-835c5ac65dd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:37.312204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b5176-752d-4ea0-99ad-835c5ac65dd3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:37.312773 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b5176-752d-4ea0-99ad-835c5ac65dd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:37.314169 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b5176-752d-4ea0-99ad-835c5ac65dd3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:37.322670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 546/1086] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Sep 7 07:43:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:37.328542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b5176-752d-4ea0-99ad-835c5ac65dd3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:37.344217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:37.346522 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98a5a3-438a-44d5-bb00-9c04d8ec0d73 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:37.383618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d45d0-292a-4512-be84-146fcd86e0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:37.384278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d45d0-292a-4512-be84-146fcd86e0dc None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:37.384642 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d45d0-292a-4512-be84-146fcd86e0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:37.384823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d45d0-292a-4512-be84-146fcd86e0dc None admin] PATH_INFO: `/v3/projects/7cfe99a9523d430381740dd9a483d06f` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:37.386021 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d45d0-292a-4512-be84-146fcd86e0dc None admin] RBAC: Authorizing `identity:get_project(project_id=7cfe99a9523d430381740dd9a483d06f)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:37.387465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d45d0-292a-4512-be84-146fcd86e0dc None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:37.389140 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 541/1087] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:43:37 2022] GET /identity/v3/projects/7cfe99a9523d430381740dd9a483d06f => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:37.394950 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d45d0-292a-4512-be84-146fcd86e0dc None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:37.429764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b5176-752d-4ea0-99ad-835c5ac65dd3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:37.445671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:37.465034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb6a79-15e8-4ac1-8c66-47db31147482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:37.465657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb6a79-15e8-4ac1-8c66-47db31147482 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:37.465937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb6a79-15e8-4ac1-8c66-47db31147482 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:37.466207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb6a79-15e8-4ac1-8c66-47db31147482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:37.477087 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb6a79-15e8-4ac1-8c66-47db31147482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:37.478513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb6a79-15e8-4ac1-8c66-47db31147482 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:37.493127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-78bb6a79-15e8-4ac1-8c66-47db31147482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:37.493127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:37.493127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:37.493127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:37.493127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:37.493127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:37.493127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:37.493127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:37.493127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:37.493127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:37.493127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:37.493127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:37.493127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:37.493127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:37.493127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:37.493127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:37.494081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:37.494975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:37.494975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:37.494975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:37.494975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 547/1088] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:37.495911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d45d0-292a-4512-be84-146fcd86e0dc None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:37.500662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb6a79-15e8-4ac1-8c66-47db31147482 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:37.531866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb56999-1c2c-449c-8c39-f0c8ac8fc262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:37.532555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb56999-1c2c-449c-8c39-f0c8ac8fc262 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:37.532835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb56999-1c2c-449c-8c39-f0c8ac8fc262 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:37.533107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb56999-1c2c-449c-8c39-f0c8ac8fc262 None admin] PATH_INFO: `/v3/projects/7cfe99a9523d430381740dd9a483d06f/users/6441d8ee1198410395397093e472f5b8/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:37.535027 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb56999-1c2c-449c-8c39-f0c8ac8fc262 None admin] RBAC: Authorizing `identity:create_grant(project_id=7cfe99a9523d430381740dd9a483d06f, user_id=6441d8ee1198410395397093e472f5b8, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:37.536496 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb56999-1c2c-449c-8c39-f0c8ac8fc262 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:37.550051 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 542/1089] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:43:37 2022] PUT /identity/v3/projects/7cfe99a9523d430381740dd9a483d06f/users/6441d8ee1198410395397093e472f5b8/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:43:37.567413 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:37.606498 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb6a79-15e8-4ac1-8c66-47db31147482 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:37.644863 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212ad65-d55e-474c-92eb-87e27c5d7e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:37.645476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212ad65-d55e-474c-92eb-87e27c5d7e27 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:37.645755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212ad65-d55e-474c-92eb-87e27c5d7e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:37.646023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212ad65-d55e-474c-92eb-87e27c5d7e27 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:37.646660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212ad65-d55e-474c-92eb-87e27c5d7e27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:37.646960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212ad65-d55e-474c-92eb-87e27c5d7e27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:37.648381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212ad65-d55e-474c-92eb-87e27c5d7e27 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:37.662415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 548/1090] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:37.668303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb56999-1c2c-449c-8c39-f0c8ac8fc262 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:37.778517 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb56999-1c2c-449c-8c39-f0c8ac8fc262 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:37.789823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:37.814750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5f6c5-33a7-460b-a4f0-2d1a036dc71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:37.815374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5f6c5-33a7-460b-a4f0-2d1a036dc71a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:37.815649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5f6c5-33a7-460b-a4f0-2d1a036dc71a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:37.815919 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5f6c5-33a7-460b-a4f0-2d1a036dc71a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:37.823738 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5f6c5-33a7-460b-a4f0-2d1a036dc71a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:37.825149 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5f6c5-33a7-460b-a4f0-2d1a036dc71a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:37.836323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-68f5f6c5-33a7-460b-a4f0-2d1a036dc71a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:43:37.836323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:37.836323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:37.836323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:37.836323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:37.836323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:37.836323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:37.836323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:37.836323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:37.836323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:37.836323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:37.836323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:37.836323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:43:37.836323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:43:37.836323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:43:37.836323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:37.837376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:37.838837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:37.838837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:37.838837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:43:37.838837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:43:37.838837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:43:37.838837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:43:37.838837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:43:37.838837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:43:37.838837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 07:43:37.838837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:37.838837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 543/1091] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 7 07:43:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:37.843792 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212ad65-d55e-474c-92eb-87e27c5d7e27 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:37.941714 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212ad65-d55e-474c-92eb-87e27c5d7e27 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:37.967505 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:37.977220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def8808-9b80-4257-923d-71acb74de4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:37.977816 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def8808-9b80-4257-923d-71acb74de4ce None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:37.978092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def8808-9b80-4257-923d-71acb74de4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:37.978361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def8808-9b80-4257-923d-71acb74de4ce None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:37.979082 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def8808-9b80-4257-923d-71acb74de4ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:37.979384 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def8808-9b80-4257-923d-71acb74de4ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:37.980856 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def8808-9b80-4257-923d-71acb74de4ce None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:37.997477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 549/1092] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 7 07:43:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:38.003297 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5f6c5-33a7-460b-a4f0-2d1a036dc71a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:38.106425 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5f6c5-33a7-460b-a4f0-2d1a036dc71a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:38.138653 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0bc6f-9441-469d-b8d2-a908fa0ee705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:38.139247 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0bc6f-9441-469d-b8d2-a908fa0ee705 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:38.140091 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0bc6f-9441-469d-b8d2-a908fa0ee705 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:38.140378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0bc6f-9441-469d-b8d2-a908fa0ee705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:38.178884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:38.207478 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0bc6f-9441-469d-b8d2-a908fa0ee705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:38.208960 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0bc6f-9441-469d-b8d2-a908fa0ee705 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:38.306873 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 544/1093] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:43:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 07:43:38.313192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def8808-9b80-4257-923d-71acb74de4ce None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:38.412851 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def8808-9b80-4257-923d-71acb74de4ce None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:38.449626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9c9e1-1cea-49b8-b1fc-1cd8d86fd33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:38.450317 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9c9e1-1cea-49b8-b1fc-1cd8d86fd33e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:38.450558 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9c9e1-1cea-49b8-b1fc-1cd8d86fd33e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:38.450827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9c9e1-1cea-49b8-b1fc-1cd8d86fd33e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:38.461273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9c9e1-1cea-49b8-b1fc-1cd8d86fd33e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:38.462684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9c9e1-1cea-49b8-b1fc-1cd8d86fd33e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:38.476821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-54e9c9e1-1cea-49b8-b1fc-1cd8d86fd33e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:38.476821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:38.476821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:38.476821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:38.476821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:38.476821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:38.476821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:38.476821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:38.476821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:38.476821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:38.476821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:38.476821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:38.476821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:43:38.476821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:43:38.476821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:43:38.476821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:38.478031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:43:38.479207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:43:38.479207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:43:38.479207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:38.480389 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 550/1094] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:38.483759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0bc6f-9441-469d-b8d2-a908fa0ee705 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:38.550782 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:38.582068 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0bc6f-9441-469d-b8d2-a908fa0ee705 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:38.614419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec5687-daba-4019-b196-810351867c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:38.615010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec5687-daba-4019-b196-810351867c95 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:38.615285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec5687-daba-4019-b196-810351867c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:38.615553 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec5687-daba-4019-b196-810351867c95 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:38.616204 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec5687-daba-4019-b196-810351867c95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:38.616503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec5687-daba-4019-b196-810351867c95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:38.617859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec5687-daba-4019-b196-810351867c95 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:38.629413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 545/1095] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:38.634960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9c9e1-1cea-49b8-b1fc-1cd8d86fd33e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:38.670384 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:38.736895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9c9e1-1cea-49b8-b1fc-1cd8d86fd33e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:38.770008 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880244a-2aec-4fba-8fed-a72822f81cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:38.770631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880244a-2aec-4fba-8fed-a72822f81cbb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:38.770906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880244a-2aec-4fba-8fed-a72822f81cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:38.771175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880244a-2aec-4fba-8fed-a72822f81cbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:38.777764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880244a-2aec-4fba-8fed-a72822f81cbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:38.779186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880244a-2aec-4fba-8fed-a72822f81cbb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:38.790658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-1880244a-2aec-4fba-8fed-a72822f81cbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:38.790658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:38.790658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:38.790658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:38.790658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:38.790658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:38.790658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:38.790658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:38.790658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:38.790658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:38.790658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:38.790658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:38.790658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:43:38.790658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:43:38.790658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:38.791734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:38.792657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:43:38.792657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:43:38.792657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:43:38.792657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:43:38.792657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:43:38.792657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:43:38.792657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:43:38.792657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:43:38.792657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 551/1096] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:43:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:43:38.798524 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec5687-daba-4019-b196-810351867c95 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:38.897623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec5687-daba-4019-b196-810351867c95 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:38.924708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:38.930821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e7f1c-4fae-4ba4-a5a6-239644a32ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:38.931418 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e7f1c-4fae-4ba4-a5a6-239644a32ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:38.931696 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e7f1c-4fae-4ba4-a5a6-239644a32ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:38.931968 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e7f1c-4fae-4ba4-a5a6-239644a32ac3 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:38.932680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e7f1c-4fae-4ba4-a5a6-239644a32ac3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:38.932984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e7f1c-4fae-4ba4-a5a6-239644a32ac3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:38.934402 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e7f1c-4fae-4ba4-a5a6-239644a32ac3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:38.946862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 546/1097] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:43:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:38.953808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880244a-2aec-4fba-8fed-a72822f81cbb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:39.044599 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:39.053804 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880244a-2aec-4fba-8fed-a72822f81cbb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:39.087624 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71a74e-cd57-4bb6-ae6e-03adf97093a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:39.088255 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71a74e-cd57-4bb6-ae6e-03adf97093a4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:39.088534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71a74e-cd57-4bb6-ae6e-03adf97093a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:39.088976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71a74e-cd57-4bb6-ae6e-03adf97093a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:39.090259 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71a74e-cd57-4bb6-ae6e-03adf97093a4 None admin] RBAC: Adding query filter params (role.id=35552dee8978468c86e59379516db233, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=81d5689e9a344bafb74ca423ffd2b42a) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:39.090811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71a74e-cd57-4bb6-ae6e-03adf97093a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:39.093544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71a74e-cd57-4bb6-ae6e-03adf97093a4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:39.101756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 552/1098] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:43:38 2022] GET /identity/v3/role_assignments?user.id=81d5689e9a344bafb74ca423ffd2b42a&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=35552dee8978468c86e59379516db233 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:39.688412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267be2cb-8e4a-4a41-8146-ecbe2d4fc87e None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:39.688906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267be2cb-8e4a-4a41-8146-ecbe2d4fc87e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:39.688906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267be2cb-8e4a-4a41-8146-ecbe2d4fc87e None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:39.690214 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 547/1099] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:43:39.695937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44c4d5-8dfa-4691-ad52-2ab1dfcba13c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:39.696410 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44c4d5-8dfa-4691-ad52-2ab1dfcba13c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:39.696511 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44c4d5-8dfa-4691-ad52-2ab1dfcba13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:39.715893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cf44c4d5-8dfa-4691-ad52-2ab1dfcba13c None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:39.717118 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cf44c4d5-8dfa-4691-ad52-2ab1dfcba13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:39.784484 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 553/1100] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:40.052027 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267be2cb-8e4a-4a41-8146-ecbe2d4fc87e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:40.152161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267be2cb-8e4a-4a41-8146-ecbe2d4fc87e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:40.185595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61af4d-92f8-477b-9045-8a2612b7dd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:40.186235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61af4d-92f8-477b-9045-8a2612b7dd77 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:40.186841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61af4d-92f8-477b-9045-8a2612b7dd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:40.187052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61af4d-92f8-477b-9045-8a2612b7dd77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:40.198644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61af4d-92f8-477b-9045-8a2612b7dd77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:43:40.203964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-cf61af4d-92f8-477b-9045-8a2612b7dd77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:43:40.203964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:43:40.203964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:43:40.203964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:43:40.203964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:43:40.203964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:43:40.203964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:43:40.203964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:43:40.203964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:43:40.203964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:43:40.203964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:43:40.203964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:43:40.203964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:43:40.203964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:43:40.203964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:43:40.203964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:43:40.204902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:43:40.204902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:43:40.204902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:43:40.204902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:43:40.204902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:43:40.204902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:43:40.204902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:43:40.204902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:43:40.204902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:43:40.204902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:43:40.204902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:43:40.204902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:43:40.204902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:43:40.204902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:43:40.204902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:43:40.204902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:43:40.205725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:43:40.205725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:43:40.205725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:43:40.205725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:43:40.205725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:43:40.205725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:43:40.205725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:43:40.205725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:43:40.205725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:43:40.205725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:43:40.205725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:43:40.205725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:43:40.205725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 548/1101] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:43:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:43:40.211800 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44c4d5-8dfa-4691-ad52-2ab1dfcba13c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:40.302993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:40.312928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44c4d5-8dfa-4691-ad52-2ab1dfcba13c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:40.348382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff0859-2b54-4df4-884a-ad17bcf5ea77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:40.348985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff0859-2b54-4df4-884a-ad17bcf5ea77 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:40.349264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff0859-2b54-4df4-884a-ad17bcf5ea77 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:40.349531 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff0859-2b54-4df4-884a-ad17bcf5ea77 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:40.350184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff0859-2b54-4df4-884a-ad17bcf5ea77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:40.350487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff0859-2b54-4df4-884a-ad17bcf5ea77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:40.351849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff0859-2b54-4df4-884a-ad17bcf5ea77 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:40.361290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 554/1102] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:43:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:40.367268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61af4d-92f8-477b-9045-8a2612b7dd77 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:40.424589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:40.474599 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61af4d-92f8-477b-9045-8a2612b7dd77 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:40.508672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07295a13-dcea-4044-abbd-43bd9fa3e9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:40.509291 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07295a13-dcea-4044-abbd-43bd9fa3e9c1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:40.509568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07295a13-dcea-4044-abbd-43bd9fa3e9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:40.509839 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07295a13-dcea-4044-abbd-43bd9fa3e9c1 None admin] PATH_INFO: `/v3/users/6441d8ee1198410395397093e472f5b8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:40.510949 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07295a13-dcea-4044-abbd-43bd9fa3e9c1 None admin] RBAC: Authorizing `identity:get_user(user_id=6441d8ee1198410395397093e472f5b8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:40.512415 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07295a13-dcea-4044-abbd-43bd9fa3e9c1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:40.513987 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 549/1103] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:43:40 2022] GET /identity/v3/users/6441d8ee1198410395397093e472f5b8 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:40.519872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff0859-2b54-4df4-884a-ad17bcf5ea77 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:40.616302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff0859-2b54-4df4-884a-ad17bcf5ea77 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:40.647443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175ab00-9a22-4663-a997-598185b670eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:40.648032 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175ab00-9a22-4663-a997-598185b670eb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:40.648337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175ab00-9a22-4663-a997-598185b670eb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:40.648606 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175ab00-9a22-4663-a997-598185b670eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:40.669632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:40.714421 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175ab00-9a22-4663-a997-598185b670eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:40.715840 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175ab00-9a22-4663-a997-598185b670eb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:40.810182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 555/1104] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:43:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:43:40.816762 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07295a13-dcea-4044-abbd-43bd9fa3e9c1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:40.913786 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07295a13-dcea-4044-abbd-43bd9fa3e9c1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:40.931662 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:40.948663 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f593c6a-20da-4708-aae2-170c2fba3a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:40.949284 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f593c6a-20da-4708-aae2-170c2fba3a0e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:40.949560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f593c6a-20da-4708-aae2-170c2fba3a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:40.949831 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f593c6a-20da-4708-aae2-170c2fba3a0e None admin] PATH_INFO: `/v3/projects/7cfe99a9523d430381740dd9a483d06f` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:40.950929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f593c6a-20da-4708-aae2-170c2fba3a0e None admin] RBAC: Authorizing `identity:get_project(project_id=7cfe99a9523d430381740dd9a483d06f)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:40.952926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f593c6a-20da-4708-aae2-170c2fba3a0e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:40.954533 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 550/1105] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:43:40 2022] GET /identity/v3/projects/7cfe99a9523d430381740dd9a483d06f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:40.960676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175ab00-9a22-4663-a997-598185b670eb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:41.010808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:41.056191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175ab00-9a22-4663-a997-598185b670eb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:41.089183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f881b4-b501-4eb0-8979-d2f28afb9820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:41.089770 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f881b4-b501-4eb0-8979-d2f28afb9820 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:41.090047 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f881b4-b501-4eb0-8979-d2f28afb9820 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:41.090316 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f881b4-b501-4eb0-8979-d2f28afb9820 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:41.091030 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f881b4-b501-4eb0-8979-d2f28afb9820 None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=7cfe99a9523d430381740dd9a483d06f, user.id=6441d8ee1198410395397093e472f5b8) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:41.091335 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f881b4-b501-4eb0-8979-d2f28afb9820 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:41.092854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f881b4-b501-4eb0-8979-d2f28afb9820 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:41.098554 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 556/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:43:40 2022] GET /identity/v3/role_assignments?user.id=6441d8ee1198410395397093e472f5b8&scope.project.id=7cfe99a9523d430381740dd9a483d06f&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:43:43.237799 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840eb2ee-4f1d-4663-b73f-99f8eb3c9f21 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:43.238355 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840eb2ee-4f1d-4663-b73f-99f8eb3c9f21 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:43.238355 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840eb2ee-4f1d-4663-b73f-99f8eb3c9f21 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:43.239674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 551/1107] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:43.244893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051a0ff-213f-45fd-8f1a-b2c046fd3e91 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:43.245260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051a0ff-213f-45fd-8f1a-b2c046fd3e91 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:43.245427 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051a0ff-213f-45fd-8f1a-b2c046fd3e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:43.263583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8051a0ff-213f-45fd-8f1a-b2c046fd3e91 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:43.264812 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8051a0ff-213f-45fd-8f1a-b2c046fd3e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:43.332041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 557/1108] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:43.600468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840eb2ee-4f1d-4663-b73f-99f8eb3c9f21 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:43.700657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840eb2ee-4f1d-4663-b73f-99f8eb3c9f21 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:43.734184 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48884953-acf3-4f0a-aedc-3106e0cc4244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:43.734795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48884953-acf3-4f0a-aedc-3106e0cc4244 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:43.735064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48884953-acf3-4f0a-aedc-3106e0cc4244 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:43.735326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48884953-acf3-4f0a-aedc-3106e0cc4244 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:43.736446 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48884953-acf3-4f0a-aedc-3106e0cc4244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:43.737862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48884953-acf3-4f0a-aedc-3106e0cc4244 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:43.739414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 552/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:43.745140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051a0ff-213f-45fd-8f1a-b2c046fd3e91 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:43.789026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:43.841181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051a0ff-213f-45fd-8f1a-b2c046fd3e91 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:43.874123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff4517-a218-445a-9528-2bf641834b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:43.874641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff4517-a218-445a-9528-2bf641834b96 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:43.874909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff4517-a218-445a-9528-2bf641834b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:43.875173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff4517-a218-445a-9528-2bf641834b96 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:43.875814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff4517-a218-445a-9528-2bf641834b96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:43.877245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff4517-a218-445a-9528-2bf641834b96 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:43.898365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 558/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:43:43 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:43:46.077241 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab1423-60fc-4edf-bcf2-943a5590d104 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:46.077761 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab1423-60fc-4edf-bcf2-943a5590d104 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:46.077853 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab1423-60fc-4edf-bcf2-943a5590d104 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:46.079149 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 553/1111] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:46.084438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaab470-2464-49b1-a1ac-c79e1444a0c8 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:46.084803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaab470-2464-49b1-a1ac-c79e1444a0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:46.084984 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaab470-2464-49b1-a1ac-c79e1444a0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:46.103207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-aaaab470-2464-49b1-a1ac-c79e1444a0c8 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:46.104418 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-aaaab470-2464-49b1-a1ac-c79e1444a0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:46.168520 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 559/1112] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:46.435781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab1423-60fc-4edf-bcf2-943a5590d104 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:46.493757 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:46.540066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab1423-60fc-4edf-bcf2-943a5590d104 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:46.574096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53a785-f74d-4457-a238-dcf4bd0fe849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:46.574736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53a785-f74d-4457-a238-dcf4bd0fe849 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:46.575015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53a785-f74d-4457-a238-dcf4bd0fe849 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:46.575288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53a785-f74d-4457-a238-dcf4bd0fe849 None admin] PATH_INFO: `/v3/roles/84a30faea3e94eb48b03ec7ceb7c1392` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:46.577312 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53a785-f74d-4457-a238-dcf4bd0fe849 None admin] RBAC: Authorizing `identity:get_role(role_id=84a30faea3e94eb48b03ec7ceb7c1392)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:46.579011 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53a785-f74d-4457-a238-dcf4bd0fe849 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:46.580349 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 554/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:43:46 2022] GET /identity/v3/roles/84a30faea3e94eb48b03ec7ceb7c1392 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:46.586085 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaab470-2464-49b1-a1ac-c79e1444a0c8 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:46.615645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:46.683713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaab470-2464-49b1-a1ac-c79e1444a0c8 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:46.718390 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d07bd-a899-48e6-a297-fd3670aa284a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:46.719013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d07bd-a899-48e6-a297-fd3670aa284a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:46.719289 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d07bd-a899-48e6-a297-fd3670aa284a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:46.719558 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d07bd-a899-48e6-a297-fd3670aa284a None admin] PATH_INFO: `/v3/users/0e062e6b60994d398efa7161a1874728` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:46.731952 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d07bd-a899-48e6-a297-fd3670aa284a None admin] RBAC: Authorizing `identity:get_user(user_id=0e062e6b60994d398efa7161a1874728)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:46.733625 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d07bd-a899-48e6-a297-fd3670aa284a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:46.735214 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 560/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:43:46 2022] GET /identity/v3/users/0e062e6b60994d398efa7161a1874728 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:46.741016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53a785-f74d-4457-a238-dcf4bd0fe849 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:46.837834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53a785-f74d-4457-a238-dcf4bd0fe849 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:46.859981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:46.871393 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0c0fd-9a7c-4ad0-a78a-b904cd51af94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:46.871982 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0c0fd-9a7c-4ad0-a78a-b904cd51af94 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:46.872287 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0c0fd-9a7c-4ad0-a78a-b904cd51af94 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:46.872560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0c0fd-9a7c-4ad0-a78a-b904cd51af94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:46.939547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0c0fd-9a7c-4ad0-a78a-b904cd51af94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:46.940977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0c0fd-9a7c-4ad0-a78a-b904cd51af94 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:47.037242 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 555/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:43:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:43:47.043147 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d07bd-a899-48e6-a297-fd3670aa284a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:47.129947 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:47.140202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d07bd-a899-48e6-a297-fd3670aa284a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:47.173651 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8f76c-3561-4a8c-9d28-70395784aee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:47.174256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8f76c-3561-4a8c-9d28-70395784aee7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:47.174526 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8f76c-3561-4a8c-9d28-70395784aee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:47.174788 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8f76c-3561-4a8c-9d28-70395784aee7 None admin] PATH_INFO: `/v3/projects/7cfe99a9523d430381740dd9a483d06f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:47.175872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8f76c-3561-4a8c-9d28-70395784aee7 None admin] RBAC: Authorizing `identity:get_project(project_id=7cfe99a9523d430381740dd9a483d06f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:47.177304 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8f76c-3561-4a8c-9d28-70395784aee7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:47.178895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 561/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:43:47 2022] GET /identity/v3/projects/7cfe99a9523d430381740dd9a483d06f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:47.185248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0c0fd-9a7c-4ad0-a78a-b904cd51af94 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:47.209787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:47.283899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0c0fd-9a7c-4ad0-a78a-b904cd51af94 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:47.317104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0d9ee-1e19-449c-b700-182c5384cabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:47.317692 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0d9ee-1e19-449c-b700-182c5384cabe None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:47.317972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0d9ee-1e19-449c-b700-182c5384cabe None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:47.318246 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0d9ee-1e19-449c-b700-182c5384cabe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:47.318952 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0d9ee-1e19-449c-b700-182c5384cabe None admin] RBAC: Adding query filter params (role.id=84a30faea3e94eb48b03ec7ceb7c1392, scope.project.id=7cfe99a9523d430381740dd9a483d06f, user.id=0e062e6b60994d398efa7161a1874728) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:47.319523 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0d9ee-1e19-449c-b700-182c5384cabe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:47.320982 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0d9ee-1e19-449c-b700-182c5384cabe None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:47.326165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 556/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:43:47 2022] GET /identity/v3/role_assignments?user.id=0e062e6b60994d398efa7161a1874728&scope.project.id=7cfe99a9523d430381740dd9a483d06f&role.id=84a30faea3e94eb48b03ec7ceb7c1392 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:49.431805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66b2b5-ea81-4fb3-af19-9dce0fa99383 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:49.432304 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66b2b5-ea81-4fb3-af19-9dce0fa99383 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:49.432415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66b2b5-ea81-4fb3-af19-9dce0fa99383 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:49.433682 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 562/1118] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:49.439449 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f693c-7e89-4570-ba46-66543db8faba None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:49.439858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f693c-7e89-4570-ba46-66543db8faba None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:49.440026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f693c-7e89-4570-ba46-66543db8faba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:49.458839 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-2b3f693c-7e89-4570-ba46-66543db8faba None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:49.460020 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2b3f693c-7e89-4570-ba46-66543db8faba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:49.525316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 557/1119] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:49.796765 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66b2b5-ea81-4fb3-af19-9dce0fa99383 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:49.893358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66b2b5-ea81-4fb3-af19-9dce0fa99383 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:49.925909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b0447-f9ad-4840-82d2-b02091eb1581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:49.926517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b0447-f9ad-4840-82d2-b02091eb1581 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:49.926786 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b0447-f9ad-4840-82d2-b02091eb1581 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:49.927048 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b0447-f9ad-4840-82d2-b02091eb1581 None admin] PATH_INFO: `/v3/roles/84a30faea3e94eb48b03ec7ceb7c1392` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:49.928424 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b0447-f9ad-4840-82d2-b02091eb1581 None admin] RBAC: Authorizing `identity:get_role(role_id=84a30faea3e94eb48b03ec7ceb7c1392)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:49.929771 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b0447-f9ad-4840-82d2-b02091eb1581 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:49.931050 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 563/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:43:49 2022] GET /identity/v3/roles/84a30faea3e94eb48b03ec7ceb7c1392 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:49.937058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f693c-7e89-4570-ba46-66543db8faba None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:49.954726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:50.033277 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f693c-7e89-4570-ba46-66543db8faba None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:50.065513 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03a17e-af6e-4c78-be4f-4aa8bd760be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:50.066059 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03a17e-af6e-4c78-be4f-4aa8bd760be6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:50.066331 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03a17e-af6e-4c78-be4f-4aa8bd760be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:50.066597 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03a17e-af6e-4c78-be4f-4aa8bd760be6 None admin] PATH_INFO: `/v3/users/0e062e6b60994d398efa7161a1874728` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:50.067690 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03a17e-af6e-4c78-be4f-4aa8bd760be6 None admin] RBAC: Authorizing `identity:get_user(user_id=0e062e6b60994d398efa7161a1874728)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:50.069135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03a17e-af6e-4c78-be4f-4aa8bd760be6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:50.070952 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 558/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:43:49 2022] GET /identity/v3/users/0e062e6b60994d398efa7161a1874728 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:50.077278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b0447-f9ad-4840-82d2-b02091eb1581 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:50.104437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:50.176918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b0447-f9ad-4840-82d2-b02091eb1581 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:50.210203 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5634d-fead-494c-b427-d544159d1102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:50.210818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5634d-fead-494c-b427-d544159d1102 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:50.211095 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5634d-fead-494c-b427-d544159d1102 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:50.211372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5634d-fead-494c-b427-d544159d1102 None admin] PATH_INFO: `/v3/projects/7cfe99a9523d430381740dd9a483d06f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:50.212499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5634d-fead-494c-b427-d544159d1102 None admin] RBAC: Authorizing `identity:get_project(project_id=7cfe99a9523d430381740dd9a483d06f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:50.213918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5634d-fead-494c-b427-d544159d1102 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:50.215524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 564/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:43:50 2022] GET /identity/v3/projects/7cfe99a9523d430381740dd9a483d06f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:50.221821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03a17e-af6e-4c78-be4f-4aa8bd760be6 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:50.303983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:50.319923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03a17e-af6e-4c78-be4f-4aa8bd760be6 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:50.352355 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d075df-67fd-4dd0-9b4d-b8e2ac42213d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:50.353000 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d075df-67fd-4dd0-9b4d-b8e2ac42213d None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:50.353279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d075df-67fd-4dd0-9b4d-b8e2ac42213d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:50.353550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d075df-67fd-4dd0-9b4d-b8e2ac42213d None admin] PATH_INFO: `/v3/projects/7cfe99a9523d430381740dd9a483d06f/users/0e062e6b60994d398efa7161a1874728/roles/84a30faea3e94eb48b03ec7ceb7c1392` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:50.355623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d075df-67fd-4dd0-9b4d-b8e2ac42213d None admin] RBAC: Authorizing `identity:create_grant(project_id=7cfe99a9523d430381740dd9a483d06f, user_id=0e062e6b60994d398efa7161a1874728, role_id=84a30faea3e94eb48b03ec7ceb7c1392)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:50.357087 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d075df-67fd-4dd0-9b4d-b8e2ac42213d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:50.375389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 559/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:43:50 2022] PUT /identity/v3/projects/7cfe99a9523d430381740dd9a483d06f/users/0e062e6b60994d398efa7161a1874728/roles/84a30faea3e94eb48b03ec7ceb7c1392 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:43:52.496175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba94c5f-bea5-47bc-8d83-8c2bf4195a4d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:52.496688 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba94c5f-bea5-47bc-8d83-8c2bf4195a4d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:52.496800 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba94c5f-bea5-47bc-8d83-8c2bf4195a4d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:52.498042 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 565/1124] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:52.503734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8b237-e8f9-434c-95e7-a2bcb165c13b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:52.504173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8b237-e8f9-434c-95e7-a2bcb165c13b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:52.504297 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8b237-e8f9-434c-95e7-a2bcb165c13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:52.522208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ecc8b237-e8f9-434c-95e7-a2bcb165c13b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:52.523380 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ecc8b237-e8f9-434c-95e7-a2bcb165c13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:52.587655 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 560/1125] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:52.861181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba94c5f-bea5-47bc-8d83-8c2bf4195a4d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:52.952723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:52.960673 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba94c5f-bea5-47bc-8d83-8c2bf4195a4d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:52.994121 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954bb4d-1fa4-42de-9790-46469428da11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:52.994738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954bb4d-1fa4-42de-9790-46469428da11 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:52.995017 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954bb4d-1fa4-42de-9790-46469428da11 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:52.995290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954bb4d-1fa4-42de-9790-46469428da11 None admin] PATH_INFO: `/v3/roles/84a30faea3e94eb48b03ec7ceb7c1392` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:52.996687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954bb4d-1fa4-42de-9790-46469428da11 None admin] RBAC: Authorizing `identity:get_role(role_id=84a30faea3e94eb48b03ec7ceb7c1392)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:52.998040 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954bb4d-1fa4-42de-9790-46469428da11 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:52.999637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 566/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:43:52 2022] GET /identity/v3/roles/84a30faea3e94eb48b03ec7ceb7c1392 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:53.005337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8b237-e8f9-434c-95e7-a2bcb165c13b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:53.079137 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:53.104502 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8b237-e8f9-434c-95e7-a2bcb165c13b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:53.136867 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d15972a-0226-428c-878d-9d42ac5be95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:53.137487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15972a-0226-428c-878d-9d42ac5be95f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:53.137762 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15972a-0226-428c-878d-9d42ac5be95f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:53.138031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15972a-0226-428c-878d-9d42ac5be95f None admin] PATH_INFO: `/v3/users/0e062e6b60994d398efa7161a1874728` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:53.139656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d15972a-0226-428c-878d-9d42ac5be95f None admin] RBAC: Authorizing `identity:get_user(user_id=0e062e6b60994d398efa7161a1874728)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:53.141109 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d15972a-0226-428c-878d-9d42ac5be95f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:53.142695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 561/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:43:53 2022] GET /identity/v3/users/0e062e6b60994d398efa7161a1874728 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:53.148505 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954bb4d-1fa4-42de-9790-46469428da11 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:53.244071 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954bb4d-1fa4-42de-9790-46469428da11 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:53.275646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a4b48-beb2-4a50-9d00-b45397550618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:53.276251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a4b48-beb2-4a50-9d00-b45397550618 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:53.276526 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a4b48-beb2-4a50-9d00-b45397550618 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:53.276793 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a4b48-beb2-4a50-9d00-b45397550618 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:53.304920 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:53.342707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a4b48-beb2-4a50-9d00-b45397550618 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:53.344145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a4b48-beb2-4a50-9d00-b45397550618 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:53.445008 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 567/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:43:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:43:53.450771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d15972a-0226-428c-878d-9d42ac5be95f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:53.547244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d15972a-0226-428c-878d-9d42ac5be95f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:53.580494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c81bd-3836-4a82-97a7-78a2fed55202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:53.581096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c81bd-3836-4a82-97a7-78a2fed55202 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:53.581368 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c81bd-3836-4a82-97a7-78a2fed55202 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:53.581634 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c81bd-3836-4a82-97a7-78a2fed55202 None admin] PATH_INFO: `/v3/projects/7cfe99a9523d430381740dd9a483d06f` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:53.582720 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c81bd-3836-4a82-97a7-78a2fed55202 None admin] RBAC: Authorizing `identity:get_project(project_id=7cfe99a9523d430381740dd9a483d06f)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:53.584160 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c81bd-3836-4a82-97a7-78a2fed55202 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:53.585751 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 562/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:43:53 2022] GET /identity/v3/projects/7cfe99a9523d430381740dd9a483d06f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:53.591780 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a4b48-beb2-4a50-9d00-b45397550618 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:53.650597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:53.688526 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a4b48-beb2-4a50-9d00-b45397550618 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:53.721713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb26a0-0166-485d-a120-5e0d7f584035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:53.722234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb26a0-0166-485d-a120-5e0d7f584035 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:53.722507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb26a0-0166-485d-a120-5e0d7f584035 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:53.722772 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb26a0-0166-485d-a120-5e0d7f584035 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:53.723478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb26a0-0166-485d-a120-5e0d7f584035 None admin] RBAC: Adding query filter params (role.id=84a30faea3e94eb48b03ec7ceb7c1392, scope.project.id=7cfe99a9523d430381740dd9a483d06f, user.id=0e062e6b60994d398efa7161a1874728) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:43:53.723778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb26a0-0166-485d-a120-5e0d7f584035 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:53.725291 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb26a0-0166-485d-a120-5e0d7f584035 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:53.730763 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 568/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:43:53 2022] GET /identity/v3/role_assignments?user.id=0e062e6b60994d398efa7161a1874728&scope.project.id=7cfe99a9523d430381740dd9a483d06f&role.id=84a30faea3e94eb48b03ec7ceb7c1392 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:55.880790 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed9dc4-ca3f-4291-92c9-61c7bff72790 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:55.881313 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed9dc4-ca3f-4291-92c9-61c7bff72790 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:55.881313 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed9dc4-ca3f-4291-92c9-61c7bff72790 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:55.882687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 563/1131] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:55.887903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2c1fd-fc1e-4f3e-ae5b-036b1518b85a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:55.888334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2c1fd-fc1e-4f3e-ae5b-036b1518b85a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:55.888486 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2c1fd-fc1e-4f3e-ae5b-036b1518b85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:55.906950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-39d2c1fd-fc1e-4f3e-ae5b-036b1518b85a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:55.908170 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-39d2c1fd-fc1e-4f3e-ae5b-036b1518b85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:55.974516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 569/1132] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 07:43:56.242572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed9dc4-ca3f-4291-92c9-61c7bff72790 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:56.255530 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:56.343983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed9dc4-ca3f-4291-92c9-61c7bff72790 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:56.376492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859554a-1962-4a6a-9d9a-2df826622249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:56.377100 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859554a-1962-4a6a-9d9a-2df826622249 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:56.377408 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859554a-1962-4a6a-9d9a-2df826622249 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:56.377685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859554a-1962-4a6a-9d9a-2df826622249 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:56.378795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2859554a-1962-4a6a-9d9a-2df826622249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:56.380256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2859554a-1962-4a6a-9d9a-2df826622249 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:56.381821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 564/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:56.387555 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2c1fd-fc1e-4f3e-ae5b-036b1518b85a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:56.442172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:56.484703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2c1fd-fc1e-4f3e-ae5b-036b1518b85a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:56.517791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913579a-df7b-4fdf-ac98-047b694d724a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:56.518323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913579a-df7b-4fdf-ac98-047b694d724a None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:56.518596 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913579a-df7b-4fdf-ac98-047b694d724a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:56.518865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913579a-df7b-4fdf-ac98-047b694d724a None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:56.519506 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913579a-df7b-4fdf-ac98-047b694d724a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:56.520940 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913579a-df7b-4fdf-ac98-047b694d724a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:56.547330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 570/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 7 07:43:56 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:43:58.768934 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358703cb-d66f-4b0a-b7ae-c39126b48e50 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:58.769432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358703cb-d66f-4b0a-b7ae-c39126b48e50 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:58.769432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358703cb-d66f-4b0a-b7ae-c39126b48e50 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:58.770732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 565/1135] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:43:58.775865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64ea56-2281-4f22-ae65-ae682e6b1213 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:58.776294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64ea56-2281-4f22-ae65-ae682e6b1213 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:58.776459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64ea56-2281-4f22-ae65-ae682e6b1213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:58.794631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ad64ea56-2281-4f22-ae65-ae682e6b1213 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:43:58.796412 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ad64ea56-2281-4f22-ae65-ae682e6b1213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:43:58.861817 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 571/1136] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:43:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:43:59.133723 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358703cb-d66f-4b0a-b7ae-c39126b48e50 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:59.211310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:59.233739 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358703cb-d66f-4b0a-b7ae-c39126b48e50 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:59.266394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143cc53-db45-43ed-b0f5-8c44847d5df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:59.267002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143cc53-db45-43ed-b0f5-8c44847d5df8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:59.267277 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143cc53-db45-43ed-b0f5-8c44847d5df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:59.267544 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143cc53-db45-43ed-b0f5-8c44847d5df8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:59.268652 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b143cc53-db45-43ed-b0f5-8c44847d5df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:59.270074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b143cc53-db45-43ed-b0f5-8c44847d5df8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:59.271612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 566/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:43:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:43:59.277220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64ea56-2281-4f22-ae65-ae682e6b1213 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:43:59.320379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:43:59.378884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64ea56-2281-4f22-ae65-ae682e6b1213 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:43:59.411706 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd322de-3df1-4023-b423-e2876621c5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:43:59.412270 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd322de-3df1-4023-b423-e2876621c5cf None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:43:59.412553 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd322de-3df1-4023-b423-e2876621c5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:43:59.412818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd322de-3df1-4023-b423-e2876621c5cf None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:43:59.413458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd322de-3df1-4023-b423-e2876621c5cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:43:59.414864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd322de-3df1-4023-b423-e2876621c5cf None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:43:59.436331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 572/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:43:59 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:44:01.658424 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89851364-d5d3-4bef-b939-5cc2027232b6 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:01.658937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89851364-d5d3-4bef-b939-5cc2027232b6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:01.658937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89851364-d5d3-4bef-b939-5cc2027232b6 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:01.660248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 567/1139] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:44:01.665533 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150e6ea-9cf7-4fe4-bf41-90e665160f64 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:01.665929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150e6ea-9cf7-4fe4-bf41-90e665160f64 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:01.666080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150e6ea-9cf7-4fe4-bf41-90e665160f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:01.684424 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8150e6ea-9cf7-4fe4-bf41-90e665160f64 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:44:01.686157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8150e6ea-9cf7-4fe4-bf41-90e665160f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:44:01.750597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 573/1140] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:44:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:44:02.022893 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89851364-d5d3-4bef-b939-5cc2027232b6 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:02.130515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89851364-d5d3-4bef-b939-5cc2027232b6 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:02.162545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a66cb2-00a7-4aab-9734-d85fc8755615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:02.163162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a66cb2-00a7-4aab-9734-d85fc8755615 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:02.163437 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a66cb2-00a7-4aab-9734-d85fc8755615 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:02.163705 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a66cb2-00a7-4aab-9734-d85fc8755615 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:02.174058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a66cb2-00a7-4aab-9734-d85fc8755615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:44:02.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-d7a66cb2-00a7-4aab-9734-d85fc8755615 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:44:02.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:44:02.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:44:02.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:44:02.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:44:02.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:44:02.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:44:02.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:44:02.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:44:02.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:44:02.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:44:02.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:44:02.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:44:02.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:44:02.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:44:02.178813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:44:02.179895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:44:02.179895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:44:02.179895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:44:02.179895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:44:02.179895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:44:02.179895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:44:02.179895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:44:02.179895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:44:02.179895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:44:02.179895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:44:02.179895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:44:02.179895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:44:02.179895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:44:02.179895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:44:02.179895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:44:02.179895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:44:02.180953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:44:02.180953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:44:02.180953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:44:02.180953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:44:02.180953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:44:02.180953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:44:02.180953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:44:02.180953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:44:02.180953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:44:02.180953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:44:02.180953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:44:02.180953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:44:02.180953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 568/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:44:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:44:02.186705 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150e6ea-9cf7-4fe4-bf41-90e665160f64 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:02.210380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:02.283117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150e6ea-9cf7-4fe4-bf41-90e665160f64 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:02.315900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf20c45-7429-4632-9e7e-834da78bc873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:02.316521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf20c45-7429-4632-9e7e-834da78bc873 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:02.316796 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf20c45-7429-4632-9e7e-834da78bc873 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:02.317064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf20c45-7429-4632-9e7e-834da78bc873 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:02.317705 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf20c45-7429-4632-9e7e-834da78bc873 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:44:02.318001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf20c45-7429-4632-9e7e-834da78bc873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:02.319599 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf20c45-7429-4632-9e7e-834da78bc873 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:02.327426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 574/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:44:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:02.333263 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a66cb2-00a7-4aab-9734-d85fc8755615 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:02.343100 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:02.432565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a66cb2-00a7-4aab-9734-d85fc8755615 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:02.465630 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19bc5e-3545-41ef-97b2-1a8d1fcd3b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:02.466248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19bc5e-3545-41ef-97b2-1a8d1fcd3b4f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:02.466526 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19bc5e-3545-41ef-97b2-1a8d1fcd3b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:02.467296 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19bc5e-3545-41ef-97b2-1a8d1fcd3b4f None admin] PATH_INFO: `/v3/users/9c402b65cf7241a28d993847439715c6` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:02.480042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b19bc5e-3545-41ef-97b2-1a8d1fcd3b4f None admin] RBAC: Authorizing `identity:get_user(user_id=9c402b65cf7241a28d993847439715c6)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:02.481508 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b19bc5e-3545-41ef-97b2-1a8d1fcd3b4f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:02.483674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 569/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:44:02 2022] GET /identity/v3/users/9c402b65cf7241a28d993847439715c6 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:02.489443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf20c45-7429-4632-9e7e-834da78bc873 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:02.570662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:02.585004 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf20c45-7429-4632-9e7e-834da78bc873 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:02.616775 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3048b1-6c0c-4fac-b704-175865274de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:02.617357 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3048b1-6c0c-4fac-b704-175865274de7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:02.617631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3048b1-6c0c-4fac-b704-175865274de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:02.617898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3048b1-6c0c-4fac-b704-175865274de7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:02.681335 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3048b1-6c0c-4fac-b704-175865274de7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:02.682743 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3048b1-6c0c-4fac-b704-175865274de7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:02.762878 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:02.778301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 575/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:44:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:44:02.784767 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19bc5e-3545-41ef-97b2-1a8d1fcd3b4f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:02.849755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:02.886039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19bc5e-3545-41ef-97b2-1a8d1fcd3b4f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:02.919205 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b089d-0827-4e45-8e42-cdc5f7adb051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:02.920416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b089d-0827-4e45-8e42-cdc5f7adb051 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:02.920629 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b089d-0827-4e45-8e42-cdc5f7adb051 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:02.920899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b089d-0827-4e45-8e42-cdc5f7adb051 None admin] PATH_INFO: `/v3/projects/88960c4e150d4391b413169d8286ae18` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:02.921995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b089d-0827-4e45-8e42-cdc5f7adb051 None admin] RBAC: Authorizing `identity:get_project(project_id=88960c4e150d4391b413169d8286ae18)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:02.923417 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b089d-0827-4e45-8e42-cdc5f7adb051 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:02.925065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 570/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:44:02 2022] GET /identity/v3/projects/88960c4e150d4391b413169d8286ae18 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:02.931242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3048b1-6c0c-4fac-b704-175865274de7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:03.026277 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3048b1-6c0c-4fac-b704-175865274de7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:03.058706 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825073ae-4a86-42eb-9e78-b4ad6d2a3fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:03.059291 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825073ae-4a86-42eb-9e78-b4ad6d2a3fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:03.059566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825073ae-4a86-42eb-9e78-b4ad6d2a3fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:03.059834 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825073ae-4a86-42eb-9e78-b4ad6d2a3fe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:03.061279 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825073ae-4a86-42eb-9e78-b4ad6d2a3fe8 None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=88960c4e150d4391b413169d8286ae18, user.id=9c402b65cf7241a28d993847439715c6) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:44:03.061585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825073ae-4a86-42eb-9e78-b4ad6d2a3fe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:03.062974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825073ae-4a86-42eb-9e78-b4ad6d2a3fe8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:03.068103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 576/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:44:02 2022] GET /identity/v3/role_assignments?user.id=9c402b65cf7241a28d993847439715c6&scope.project.id=88960c4e150d4391b413169d8286ae18&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:44:05.196603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c178e68-b133-4433-8e33-9d78214c41ef None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:05.197161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c178e68-b133-4433-8e33-9d78214c41ef None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:05.197161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c178e68-b133-4433-8e33-9d78214c41ef None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:05.198468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 571/1147] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:44:05.203537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9946ad1-d2d0-4e43-9c17-6eace8968e9c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:05.203935 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9946ad1-d2d0-4e43-9c17-6eace8968e9c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:05.204114 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9946ad1-d2d0-4e43-9c17-6eace8968e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:05.222190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d9946ad1-d2d0-4e43-9c17-6eace8968e9c None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:44:05.223367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d9946ad1-d2d0-4e43-9c17-6eace8968e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:44:05.268382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:05.289157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 577/1148] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:44:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:44:05.566049 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c178e68-b133-4433-8e33-9d78214c41ef None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:05.665590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c178e68-b133-4433-8e33-9d78214c41ef None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:05.693267 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:05.698758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154e0f1-99eb-480b-957a-43eb5a9f7f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:05.699376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154e0f1-99eb-480b-957a-43eb5a9f7f1a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:05.699649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154e0f1-99eb-480b-957a-43eb5a9f7f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:05.699916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154e0f1-99eb-480b-957a-43eb5a9f7f1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:05.710456 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154e0f1-99eb-480b-957a-43eb5a9f7f1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:44:05.715201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-c154e0f1-99eb-480b-957a-43eb5a9f7f1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:44:05.715201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:44:05.715201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:44:05.715201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:44:05.715201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:44:05.715201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:44:05.715201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:44:05.715201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:44:05.715201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:44:05.715201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:44:05.715201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:44:05.715201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:44:05.715201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:44:05.715201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:44:05.715201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:44:05.715201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:44:05.716382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:44:05.716382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:44:05.716382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:44:05.716382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:44:05.716382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:44:05.716382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:44:05.716382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:44:05.716382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:44:05.716382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:44:05.716382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:44:05.716382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:44:05.716382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:44:05.716382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:44:05.716382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:44:05.716382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:44:05.716382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:44:05.718350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:44:05.718350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:44:05.718350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:44:05.718350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:44:05.718350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:44:05.718350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:44:05.718350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:44:05.718350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:44:05.718350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:44:05.718350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:44:05.718350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:44:05.718350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:44:05.718350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 572/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:44:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:44:05.722931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9946ad1-d2d0-4e43-9c17-6eace8968e9c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:05.823037 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9946ad1-d2d0-4e43-9c17-6eace8968e9c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:05.857632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e031a5-b135-4af5-8fac-044bb1a18cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:05.858213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e031a5-b135-4af5-8fac-044bb1a18cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:05.858485 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e031a5-b135-4af5-8fac-044bb1a18cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:05.858750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e031a5-b135-4af5-8fac-044bb1a18cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:05.859393 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e031a5-b135-4af5-8fac-044bb1a18cc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:44:05.859688 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e031a5-b135-4af5-8fac-044bb1a18cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:05.861062 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e031a5-b135-4af5-8fac-044bb1a18cc4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:05.868696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 578/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:44:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:05.874730 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154e0f1-99eb-480b-957a-43eb5a9f7f1a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:05.975356 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154e0f1-99eb-480b-957a-43eb5a9f7f1a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:06.009222 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469aa61-431a-49bd-a5fa-bc50185adbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:06.009833 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469aa61-431a-49bd-a5fa-bc50185adbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:06.010107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469aa61-431a-49bd-a5fa-bc50185adbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:06.010378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469aa61-431a-49bd-a5fa-bc50185adbf1 None admin] PATH_INFO: `/v3/users/9c402b65cf7241a28d993847439715c6` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:06.011484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469aa61-431a-49bd-a5fa-bc50185adbf1 None admin] RBAC: Authorizing `identity:get_user(user_id=9c402b65cf7241a28d993847439715c6)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:06.012942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469aa61-431a-49bd-a5fa-bc50185adbf1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:06.015193 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 573/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:44:05 2022] GET /identity/v3/users/9c402b65cf7241a28d993847439715c6 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:06.020943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e031a5-b135-4af5-8fac-044bb1a18cc4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:06.063662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:06.119248 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e031a5-b135-4af5-8fac-044bb1a18cc4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:06.152468 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df480ad-b4c8-4106-9fcb-5ab06849d167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:06.153088 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df480ad-b4c8-4106-9fcb-5ab06849d167 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:06.153365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df480ad-b4c8-4106-9fcb-5ab06849d167 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:06.153634 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df480ad-b4c8-4106-9fcb-5ab06849d167 None admin] PATH_INFO: `/v3/projects/88960c4e150d4391b413169d8286ae18` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:06.155482 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df480ad-b4c8-4106-9fcb-5ab06849d167 None admin] RBAC: Authorizing `identity:get_project(project_id=88960c4e150d4391b413169d8286ae18)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:06.156933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df480ad-b4c8-4106-9fcb-5ab06849d167 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:06.158552 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 579/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:44:06 2022] GET /identity/v3/projects/88960c4e150d4391b413169d8286ae18 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:06.164459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469aa61-431a-49bd-a5fa-bc50185adbf1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:06.222158 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:06.261397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469aa61-431a-49bd-a5fa-bc50185adbf1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:06.294011 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6af866-98e6-497e-9915-52672a23c9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:06.294822 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6af866-98e6-497e-9915-52672a23c9cf None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:06.295101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6af866-98e6-497e-9915-52672a23c9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:06.295368 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6af866-98e6-497e-9915-52672a23c9cf None admin] PATH_INFO: `/v3/projects/88960c4e150d4391b413169d8286ae18/users/9c402b65cf7241a28d993847439715c6/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:06.297248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6af866-98e6-497e-9915-52672a23c9cf None admin] RBAC: Authorizing `identity:create_grant(project_id=88960c4e150d4391b413169d8286ae18, user_id=9c402b65cf7241a28d993847439715c6, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:06.298660 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6af866-98e6-497e-9915-52672a23c9cf None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:06.311473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 574/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:44:06 2022] PUT /identity/v3/projects/88960c4e150d4391b413169d8286ae18/users/9c402b65cf7241a28d993847439715c6/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:44:08.421522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea0162-13f2-4750-b1d3-8cbd75438197 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:08.421998 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea0162-13f2-4750-b1d3-8cbd75438197 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:08.422097 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea0162-13f2-4750-b1d3-8cbd75438197 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:08.423381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 580/1154] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:44:08.428529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c5a72-f31d-44ed-ac5f-ecc0e8e895b9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:08.428907 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c5a72-f31d-44ed-ac5f-ecc0e8e895b9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:08.429081 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c5a72-f31d-44ed-ac5f-ecc0e8e895b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:08.447784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-038c5a72-f31d-44ed-ac5f-ecc0e8e895b9 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:44:08.449007 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-038c5a72-f31d-44ed-ac5f-ecc0e8e895b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:44:08.515442 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 575/1155] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:44:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:44:08.783510 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea0162-13f2-4750-b1d3-8cbd75438197 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:08.887216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea0162-13f2-4750-b1d3-8cbd75438197 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:08.898462 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:08.922372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ff967-3142-40b6-ae31-59f2ab717c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:08.922996 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ff967-3142-40b6-ae31-59f2ab717c0f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:08.923272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ff967-3142-40b6-ae31-59f2ab717c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:08.923542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ff967-3142-40b6-ae31-59f2ab717c0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:08.933221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ff967-3142-40b6-ae31-59f2ab717c0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:44:08.939173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-4c8ff967-3142-40b6-ae31-59f2ab717c0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:44:08.939173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:44:08.939173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:44:08.939173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:44:08.939173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:44:08.939173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:44:08.939173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:44:08.939173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:44:08.939173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:44:08.939173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:44:08.939173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:44:08.939173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:44:08.939173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:44:08.939173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:44:08.939173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:44:08.939173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:44:08.940379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:44:08.940379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:44:08.940379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:44:08.940379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:44:08.940379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:44:08.940379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:44:08.940379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:44:08.940379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:44:08.940379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:44:08.940379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:44:08.940379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:44:08.940379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:44:08.940379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:44:08.940379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:44:08.940379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:44:08.940379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:44:08.941337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:44:08.941337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:44:08.941337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:44:08.941337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:44:08.941337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:44:08.941337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:44:08.941337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:44:08.941337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:44:08.941337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:44:08.941337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:44:08.941337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:44:08.941337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:44:08.941337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 581/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:44:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:44:08.947103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c5a72-f31d-44ed-ac5f-ecc0e8e895b9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:08.995179 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:09.046770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c5a72-f31d-44ed-ac5f-ecc0e8e895b9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:09.080260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bb7e5-2ad6-44fa-a233-59c87bf8e159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:09.080931 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bb7e5-2ad6-44fa-a233-59c87bf8e159 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:09.081211 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bb7e5-2ad6-44fa-a233-59c87bf8e159 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:09.081486 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bb7e5-2ad6-44fa-a233-59c87bf8e159 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:09.082137 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527bb7e5-2ad6-44fa-a233-59c87bf8e159 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:44:09.082442 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527bb7e5-2ad6-44fa-a233-59c87bf8e159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:09.083811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527bb7e5-2ad6-44fa-a233-59c87bf8e159 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:09.092340 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 576/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:44:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:09.098089 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ff967-3142-40b6-ae31-59f2ab717c0f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:09.193150 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ff967-3142-40b6-ae31-59f2ab717c0f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:09.225470 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a203f5-5273-45b4-bb11-06375dfa5dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:09.226068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a203f5-5273-45b4-bb11-06375dfa5dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:09.226340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a203f5-5273-45b4-bb11-06375dfa5dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:09.226606 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a203f5-5273-45b4-bb11-06375dfa5dc5 None admin] PATH_INFO: `/v3/users/9c402b65cf7241a28d993847439715c6` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:09.227689 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a203f5-5273-45b4-bb11-06375dfa5dc5 None admin] RBAC: Authorizing `identity:get_user(user_id=9c402b65cf7241a28d993847439715c6)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:09.229193 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a203f5-5273-45b4-bb11-06375dfa5dc5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:09.230727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 582/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:44:09 2022] GET /identity/v3/users/9c402b65cf7241a28d993847439715c6 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:09.236937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bb7e5-2ad6-44fa-a233-59c87bf8e159 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:09.335130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bb7e5-2ad6-44fa-a233-59c87bf8e159 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:09.347744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:09.369064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30abda0-0edb-42eb-bffe-02ab557acaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:09.369649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30abda0-0edb-42eb-bffe-02ab557acaff None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:09.369921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30abda0-0edb-42eb-bffe-02ab557acaff None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:09.370186 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30abda0-0edb-42eb-bffe-02ab557acaff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:09.436072 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30abda0-0edb-42eb-bffe-02ab557acaff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:09.437544 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30abda0-0edb-42eb-bffe-02ab557acaff None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:09.532384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 577/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:44:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:44:09.538276 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a203f5-5273-45b4-bb11-06375dfa5dc5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:09.577734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:09.634598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a203f5-5273-45b4-bb11-06375dfa5dc5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:09.667195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17121a-12df-4d16-8a7b-59c5e4451c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:09.667802 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17121a-12df-4d16-8a7b-59c5e4451c57 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:09.668072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17121a-12df-4d16-8a7b-59c5e4451c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:09.668375 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17121a-12df-4d16-8a7b-59c5e4451c57 None admin] PATH_INFO: `/v3/projects/88960c4e150d4391b413169d8286ae18` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:09.669488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17121a-12df-4d16-8a7b-59c5e4451c57 None admin] RBAC: Authorizing `identity:get_project(project_id=88960c4e150d4391b413169d8286ae18)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:09.670937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17121a-12df-4d16-8a7b-59c5e4451c57 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:09.672551 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 583/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:44:09 2022] GET /identity/v3/projects/88960c4e150d4391b413169d8286ae18 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:44:09.678622 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30abda0-0edb-42eb-bffe-02ab557acaff None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:09.692230 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:09.775347 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30abda0-0edb-42eb-bffe-02ab557acaff None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:09.807941 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89cff19-f4a9-4803-b47f-ad90936c9d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:09.808562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cff19-f4a9-4803-b47f-ad90936c9d4c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:09.808838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cff19-f4a9-4803-b47f-ad90936c9d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:09.809106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cff19-f4a9-4803-b47f-ad90936c9d4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:09.809810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89cff19-f4a9-4803-b47f-ad90936c9d4c None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=88960c4e150d4391b413169d8286ae18, user.id=9c402b65cf7241a28d993847439715c6) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:44:09.810639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89cff19-f4a9-4803-b47f-ad90936c9d4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:09.812182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89cff19-f4a9-4803-b47f-ad90936c9d4c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:09.817708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 578/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:44:09 2022] GET /identity/v3/role_assignments?user.id=9c402b65cf7241a28d993847439715c6&scope.project.id=88960c4e150d4391b413169d8286ae18&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:11.966702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec650108-6f3a-4890-9a52-d56e46fdb855 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:11.967262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec650108-6f3a-4890-9a52-d56e46fdb855 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:11.967262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec650108-6f3a-4890-9a52-d56e46fdb855 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:11.968636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 584/1162] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:44:11.973758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338e909-2f94-4733-b1bc-1a331912cce2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:11.974139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338e909-2f94-4733-b1bc-1a331912cce2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:11.974295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338e909-2f94-4733-b1bc-1a331912cce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:11.992953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e338e909-2f94-4733-b1bc-1a331912cce2 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:44:11.994129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e338e909-2f94-4733-b1bc-1a331912cce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:44:12.059412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 579/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:44:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:44:12.325123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec650108-6f3a-4890-9a52-d56e46fdb855 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:12.422920 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec650108-6f3a-4890-9a52-d56e46fdb855 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:12.430070 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:12.467567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74990969-c333-4898-a923-e5bce6ac82fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:12.468314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74990969-c333-4898-a923-e5bce6ac82fe None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:12.468448 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74990969-c333-4898-a923-e5bce6ac82fe None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:12.468718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74990969-c333-4898-a923-e5bce6ac82fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:12.538087 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74990969-c333-4898-a923-e5bce6ac82fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:12.539502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74990969-c333-4898-a923-e5bce6ac82fe None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:12.632205 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 585/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:44:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:44:12.638617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338e909-2f94-4733-b1bc-1a331912cce2 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:12.646656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:12.735528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338e909-2f94-4733-b1bc-1a331912cce2 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:12.769283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16715588-0769-4cc1-9ac7-9e2f4eeb396e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:12.769899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16715588-0769-4cc1-9ac7-9e2f4eeb396e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:12.770178 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16715588-0769-4cc1-9ac7-9e2f4eeb396e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:12.770449 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16715588-0769-4cc1-9ac7-9e2f4eeb396e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:12.776955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16715588-0769-4cc1-9ac7-9e2f4eeb396e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:12.778391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16715588-0769-4cc1-9ac7-9e2f4eeb396e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-16715588-0769-4cc1-9ac7-9e2f4eeb396e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:44:12.790248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:44:12.791464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:44:12.791464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:44:12.791464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:44:12.791464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:44:12.791464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:44:12.791464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:44:12.791464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:44:12.791464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:44:12.791464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:44:12.791464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:44:12.791464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:44:12.791464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:44:12.791464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:44:12.791464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:44:12.791464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:44:12.791464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:44:12.792541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:44:12.792541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:44:12.792541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:44:12.792541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:44:12.792541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:44:12.792541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:44:12.792541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:44:12.792541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:44:12.792541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:44:12.792541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:44:12.792541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 07 07:44:12.792541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:44:12.792541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 07 07:44:12.792541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:44:12.792541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 580/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 7 07:44:12 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:44:12.798112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74990969-c333-4898-a923-e5bce6ac82fe None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:12.806032 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:12.893819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74990969-c333-4898-a923-e5bce6ac82fe None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:12.928790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf67c5d-6b6f-4a0c-84b7-79f32a492cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:12.929371 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf67c5d-6b6f-4a0c-84b7-79f32a492cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:12.929642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf67c5d-6b6f-4a0c-84b7-79f32a492cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:12.929908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf67c5d-6b6f-4a0c-84b7-79f32a492cc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:12.930525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf67c5d-6b6f-4a0c-84b7-79f32a492cc3 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:44:12.930818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf67c5d-6b6f-4a0c-84b7-79f32a492cc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:12.932171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf67c5d-6b6f-4a0c-84b7-79f32a492cc3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:12.938863 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 586/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Sep 7 07:44:12 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:15.054773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433e764-603d-4a4e-a39d-73c2fc5375a1 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:15.055274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433e764-603d-4a4e-a39d-73c2fc5375a1 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:15.055274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433e764-603d-4a4e-a39d-73c2fc5375a1 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:15.056651 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 581/1167] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:44:15.062521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60e43c-38da-471e-9f36-5857314a3cf0 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:15.062930 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60e43c-38da-471e-9f36-5857314a3cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:15.063073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60e43c-38da-471e-9f36-5857314a3cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:15.081704 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-dd60e43c-38da-471e-9f36-5857314a3cf0 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:44:15.082881 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-dd60e43c-38da-471e-9f36-5857314a3cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:44:15.142153 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:15.147089 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 587/1168] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:44:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:44:15.422110 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433e764-603d-4a4e-a39d-73c2fc5375a1 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:15.488818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:15.526410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433e764-603d-4a4e-a39d-73c2fc5375a1 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:15.559198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d04359-3eac-4d42-9c02-12f72fd63483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:15.559789 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d04359-3eac-4d42-9c02-12f72fd63483 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:15.560312 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d04359-3eac-4d42-9c02-12f72fd63483 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:15.560783 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d04359-3eac-4d42-9c02-12f72fd63483 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:15.561443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d04359-3eac-4d42-9c02-12f72fd63483 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:15.562805 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d04359-3eac-4d42-9c02-12f72fd63483 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:15.579800 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 582/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Wed Sep 7 07:44:15 2022] POST /identity/v3/domains => generated 267 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:44:17.741374 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f78ee0-9926-4174-b5b1-77665d8d1c24 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:17.741873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f78ee0-9926-4174-b5b1-77665d8d1c24 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:17.741873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f78ee0-9926-4174-b5b1-77665d8d1c24 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:17.743170 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 588/1170] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:44:17.748283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de362bd1-9d05-4d20-bfa0-9e35ebbe56a2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:17.748694 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de362bd1-9d05-4d20-bfa0-9e35ebbe56a2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:17.748822 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de362bd1-9d05-4d20-bfa0-9e35ebbe56a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:17.767436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-de362bd1-9d05-4d20-bfa0-9e35ebbe56a2 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:44:17.768669 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-de362bd1-9d05-4d20-bfa0-9e35ebbe56a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:44:17.833245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 583/1171] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:44:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:44:18.097243 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f78ee0-9926-4174-b5b1-77665d8d1c24 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:18.194692 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f78ee0-9926-4174-b5b1-77665d8d1c24 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:18.226929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93960b01-4829-4ee2-8565-692261381602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:18.227529 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93960b01-4829-4ee2-8565-692261381602 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:18.227801 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93960b01-4829-4ee2-8565-692261381602 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:18.228067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93960b01-4829-4ee2-8565-692261381602 None admin] PATH_INFO: `/v3/domains/5276d8286c83463b8960015b8ef63c72` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:18.238659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93960b01-4829-4ee2-8565-692261381602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5276d8286c83463b8960015b8ef63c72)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:18.240067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93960b01-4829-4ee2-8565-692261381602 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:18.241637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 589/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Wed Sep 7 07:44:18 2022] GET /identity/v3/domains/5276d8286c83463b8960015b8ef63c72 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:44:18.247299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de362bd1-9d05-4d20-bfa0-9e35ebbe56a2 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:18.300887 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:18.343339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de362bd1-9d05-4d20-bfa0-9e35ebbe56a2 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:18.375748 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec2567-9d41-4ea5-8270-2b7612b2c636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:18.376306 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec2567-9d41-4ea5-8270-2b7612b2c636 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:18.376579 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec2567-9d41-4ea5-8270-2b7612b2c636 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:18.376841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec2567-9d41-4ea5-8270-2b7612b2c636 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:18.377502 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec2567-9d41-4ea5-8270-2b7612b2c636 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:18.378921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec2567-9d41-4ea5-8270-2b7612b2c636 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:18.405920 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 584/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 7 07:44:18 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:44:20.596663 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ed7ce-2281-4b8c-9f78-b427e7f6a332 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:20.597172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ed7ce-2281-4b8c-9f78-b427e7f6a332 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:20.597287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ed7ce-2281-4b8c-9f78-b427e7f6a332 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:20.598544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 590/1174] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:44:20.603744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25a07a-8354-42a3-8e76-2076fc634d3d None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:20.604127 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25a07a-8354-42a3-8e76-2076fc634d3d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:20.604392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25a07a-8354-42a3-8e76-2076fc634d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:20.622827 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-2b25a07a-8354-42a3-8e76-2076fc634d3d None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:44:20.624015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2b25a07a-8354-42a3-8e76-2076fc634d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:44:20.688116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 585/1175] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:44:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:44:20.964249 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ed7ce-2281-4b8c-9f78-b427e7f6a332 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:21.023280 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:21.064723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ed7ce-2281-4b8c-9f78-b427e7f6a332 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:21.097448 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4109b7c-c66f-4d7f-a142-91f6571d0561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:21.098057 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4109b7c-c66f-4d7f-a142-91f6571d0561 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:21.098338 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4109b7c-c66f-4d7f-a142-91f6571d0561 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:21.098611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4109b7c-c66f-4d7f-a142-91f6571d0561 None admin] PATH_INFO: `/v3/domains/5276d8286c83463b8960015b8ef63c72` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:21.099720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4109b7c-c66f-4d7f-a142-91f6571d0561 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5276d8286c83463b8960015b8ef63c72)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:21.101173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4109b7c-c66f-4d7f-a142-91f6571d0561 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:21.102719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 591/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Wed Sep 7 07:44:20 2022] GET /identity/v3/domains/5276d8286c83463b8960015b8ef63c72 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:21.109013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25a07a-8354-42a3-8e76-2076fc634d3d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:21.146799 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:21.207934 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25a07a-8354-42a3-8e76-2076fc634d3d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:21.241526 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c42fea-7e8f-4102-a271-7c523d502337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:21.242060 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c42fea-7e8f-4102-a271-7c523d502337 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:21.242337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c42fea-7e8f-4102-a271-7c523d502337 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:21.242609 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c42fea-7e8f-4102-a271-7c523d502337 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:21.243240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c42fea-7e8f-4102-a271-7c523d502337 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:21.244702 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c42fea-7e8f-4102-a271-7c523d502337 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:21.264231 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 586/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:44:21 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:44:23.451051 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bf3a6-59c1-434c-a5b5-e47fd8f8d427 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:23.451565 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bf3a6-59c1-434c-a5b5-e47fd8f8d427 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:23.451673 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bf3a6-59c1-434c-a5b5-e47fd8f8d427 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:23.452950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 592/1178] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:44:23.458266 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5f6d0-9da0-4275-aa3e-9ebe921557ae None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:23.458629 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5f6d0-9da0-4275-aa3e-9ebe921557ae None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:23.458827 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5f6d0-9da0-4275-aa3e-9ebe921557ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:23.477084 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-93c5f6d0-9da0-4275-aa3e-9ebe921557ae None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:44:23.478255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-93c5f6d0-9da0-4275-aa3e-9ebe921557ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:44:23.542302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 587/1179] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:44:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:44:23.816691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bf3a6-59c1-434c-a5b5-e47fd8f8d427 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:23.914203 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bf3a6-59c1-434c-a5b5-e47fd8f8d427 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:23.939848 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:23.949549 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ac524-386d-4d34-8ce8-d20234465dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:23.950712 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ac524-386d-4d34-8ce8-d20234465dfe None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:23.950993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ac524-386d-4d34-8ce8-d20234465dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:23.951262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ac524-386d-4d34-8ce8-d20234465dfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:23.961441 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ac524-386d-4d34-8ce8-d20234465dfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:44:23.966976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-938ac524-386d-4d34-8ce8-d20234465dfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:44:23.966976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:44:23.966976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:44:23.966976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:44:23.966976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:44:23.966976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:44:23.966976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:44:23.966976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:44:23.966976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:44:23.966976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:44:23.966976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:44:23.966976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:44:23.966976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:44:23.966976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:44:23.966976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:44:23.966976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:44:23.968308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:44:23.968308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:44:23.968308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:44:23.968308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:44:23.968308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:44:23.968308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:44:23.968308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:44:23.968308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:44:23.968308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:44:23.968308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:44:23.968308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:44:23.968308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:44:23.968308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:44:23.968308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:44:23.968308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:44:23.968308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:44:23.969406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:44:23.969406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:44:23.969406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:44:23.969406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:44:23.969406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:44:23.969406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:44:23.969406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:44:23.969406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:44:23.969406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:44:23.969406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:44:23.969406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:44:23.969406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:44:23.969406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 593/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:44:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:44:23.974390 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5f6d0-9da0-4275-aa3e-9ebe921557ae None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:23.998800 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:24.074242 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5f6d0-9da0-4275-aa3e-9ebe921557ae None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:24.108616 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bda3f1-a814-42c8-97d7-37004734d557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:24.109211 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bda3f1-a814-42c8-97d7-37004734d557 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:24.109488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bda3f1-a814-42c8-97d7-37004734d557 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:24.109759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bda3f1-a814-42c8-97d7-37004734d557 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:24.110409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bda3f1-a814-42c8-97d7-37004734d557 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:44:24.110708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bda3f1-a814-42c8-97d7-37004734d557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:24.112073 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bda3f1-a814-42c8-97d7-37004734d557 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:24.119920 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 588/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:44:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:24.126465 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ac524-386d-4d34-8ce8-d20234465dfe None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:24.225331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ac524-386d-4d34-8ce8-d20234465dfe None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:24.258566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e3397-c6fe-4e31-ba35-8b03f883f43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:24.259175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e3397-c6fe-4e31-ba35-8b03f883f43e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:24.259451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e3397-c6fe-4e31-ba35-8b03f883f43e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:24.259720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e3397-c6fe-4e31-ba35-8b03f883f43e None admin] PATH_INFO: `/v3/users/df1178bd868546b1908cfc8888730953` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:24.272844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e3397-c6fe-4e31-ba35-8b03f883f43e None admin] RBAC: Authorizing `identity:get_user(user_id=df1178bd868546b1908cfc8888730953)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:24.274266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e3397-c6fe-4e31-ba35-8b03f883f43e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:24.275839 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 594/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:44:24 2022] GET /identity/v3/users/df1178bd868546b1908cfc8888730953 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:24.281764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bda3f1-a814-42c8-97d7-37004734d557 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:24.365859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:24.380767 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bda3f1-a814-42c8-97d7-37004734d557 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:24.413367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b57dc6-5ad3-4d33-b1f8-d05afdb32580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:24.413965 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b57dc6-5ad3-4d33-b1f8-d05afdb32580 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:24.414243 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b57dc6-5ad3-4d33-b1f8-d05afdb32580 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:24.414517 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b57dc6-5ad3-4d33-b1f8-d05afdb32580 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:24.480586 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b57dc6-5ad3-4d33-b1f8-d05afdb32580 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:24.482293 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b57dc6-5ad3-4d33-b1f8-d05afdb32580 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:24.562155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:24.577445 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 589/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:44:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:44:24.583385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e3397-c6fe-4e31-ba35-8b03f883f43e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:24.628329 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:24.678202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e3397-c6fe-4e31-ba35-8b03f883f43e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:24.710972 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a192661-809f-4ab7-9b32-981bc5f6a0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:24.711586 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a192661-809f-4ab7-9b32-981bc5f6a0f7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:24.711864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a192661-809f-4ab7-9b32-981bc5f6a0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:24.712160 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a192661-809f-4ab7-9b32-981bc5f6a0f7 None admin] PATH_INFO: `/v3/projects/1f7bba182b7349baa3f83745b9914b49` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:24.713260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a192661-809f-4ab7-9b32-981bc5f6a0f7 None admin] RBAC: Authorizing `identity:get_project(project_id=1f7bba182b7349baa3f83745b9914b49)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:24.714668 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a192661-809f-4ab7-9b32-981bc5f6a0f7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:24.716551 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 595/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:44:24 2022] GET /identity/v3/projects/1f7bba182b7349baa3f83745b9914b49 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:24.722582 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b57dc6-5ad3-4d33-b1f8-d05afdb32580 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:24.819887 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b57dc6-5ad3-4d33-b1f8-d05afdb32580 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:24.853305 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e72309-4bff-4990-ae8f-e8b8f2847113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:24.853884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e72309-4bff-4990-ae8f-e8b8f2847113 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:24.854160 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e72309-4bff-4990-ae8f-e8b8f2847113 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:24.854430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e72309-4bff-4990-ae8f-e8b8f2847113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:24.855131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e72309-4bff-4990-ae8f-e8b8f2847113 None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=1f7bba182b7349baa3f83745b9914b49, user.id=df1178bd868546b1908cfc8888730953) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:44:24.855435 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e72309-4bff-4990-ae8f-e8b8f2847113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:24.856907 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e72309-4bff-4990-ae8f-e8b8f2847113 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:24.862155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 590/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:44:24 2022] GET /identity/v3/role_assignments?user.id=df1178bd868546b1908cfc8888730953&scope.project.id=1f7bba182b7349baa3f83745b9914b49&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:26.964638 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a883b30-43a1-4ec2-9232-6b6282c19588 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:26.965182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a883b30-43a1-4ec2-9232-6b6282c19588 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:26.965182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a883b30-43a1-4ec2-9232-6b6282c19588 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:26.967292 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 596/1186] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:44:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:44:26.972509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55586d-4250-4c48-956a-be3243782841 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:26.972911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55586d-4250-4c48-956a-be3243782841 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:26.973059 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55586d-4250-4c48-956a-be3243782841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:26.991033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-db55586d-4250-4c48-956a-be3243782841 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:44:26.992264 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-db55586d-4250-4c48-956a-be3243782841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:44:27.038638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:27.059162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 591/1187] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:44:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:44:27.325829 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a883b30-43a1-4ec2-9232-6b6282c19588 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:27.423709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a883b30-43a1-4ec2-9232-6b6282c19588 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:27.437810 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:27.457837 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a0736-633a-466f-8f41-dab09bd6ae2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:27.458564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a0736-633a-466f-8f41-dab09bd6ae2f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:27.458843 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a0736-633a-466f-8f41-dab09bd6ae2f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:27.459113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a0736-633a-466f-8f41-dab09bd6ae2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:27.469373 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a0736-633a-466f-8f41-dab09bd6ae2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:44:27.474109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-6b6a0736-633a-466f-8f41-dab09bd6ae2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:44:27.474109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:44:27.474109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:44:27.474109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:44:27.474109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:44:27.474109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:44:27.474109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:44:27.474109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:44:27.474109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:44:27.474109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:44:27.474109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:44:27.474109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:44:27.474109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:44:27.474109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:44:27.474109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:44:27.474109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:44:27.475191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:44:27.475191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:44:27.475191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:44:27.475191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:44:27.475191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:44:27.475191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:44:27.475191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:44:27.475191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:44:27.475191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:44:27.475191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:44:27.475191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:44:27.475191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:44:27.475191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:44:27.475191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:44:27.475191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:44:27.475191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:44:27.476264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:44:27.476264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:44:27.476264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:44:27.476264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:44:27.476264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:44:27.476264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:44:27.476264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:44:27.476264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:44:27.476264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:44:27.476264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:44:27.476264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:44:27.476264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:44:27.476264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 597/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:44:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:44:27.481196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55586d-4250-4c48-956a-be3243782841 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:27.580100 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55586d-4250-4c48-956a-be3243782841 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:27.614019 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f84da7-9c41-4a43-8641-95b93eb60036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:27.614618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f84da7-9c41-4a43-8641-95b93eb60036 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:27.614897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f84da7-9c41-4a43-8641-95b93eb60036 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:27.615171 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f84da7-9c41-4a43-8641-95b93eb60036 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:27.615823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f84da7-9c41-4a43-8641-95b93eb60036 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:44:27.616148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f84da7-9c41-4a43-8641-95b93eb60036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:27.617517 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f84da7-9c41-4a43-8641-95b93eb60036 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:27.625623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 592/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:44:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:27.631439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a0736-633a-466f-8f41-dab09bd6ae2f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:27.731254 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a0736-633a-466f-8f41-dab09bd6ae2f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:27.769262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e35446-061f-4b1c-95ae-b3a7f254f5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:27.769884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e35446-061f-4b1c-95ae-b3a7f254f5bb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:27.770160 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e35446-061f-4b1c-95ae-b3a7f254f5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:27.770427 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e35446-061f-4b1c-95ae-b3a7f254f5bb None admin] PATH_INFO: `/v3/users/df1178bd868546b1908cfc8888730953` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:27.771528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e35446-061f-4b1c-95ae-b3a7f254f5bb None admin] RBAC: Authorizing `identity:get_user(user_id=df1178bd868546b1908cfc8888730953)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:27.772977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e35446-061f-4b1c-95ae-b3a7f254f5bb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:27.774566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 598/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:44:27 2022] GET /identity/v3/users/df1178bd868546b1908cfc8888730953 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:27.781219 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f84da7-9c41-4a43-8641-95b93eb60036 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:27.824238 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:27.884688 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f84da7-9c41-4a43-8641-95b93eb60036 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:27.920102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9a821-91da-46b4-9fcc-cb1b623c60de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:27.920748 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9a821-91da-46b4-9fcc-cb1b623c60de None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:27.921025 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9a821-91da-46b4-9fcc-cb1b623c60de None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:27.921293 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9a821-91da-46b4-9fcc-cb1b623c60de None admin] PATH_INFO: `/v3/projects/1f7bba182b7349baa3f83745b9914b49` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:27.922401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9a821-91da-46b4-9fcc-cb1b623c60de None admin] RBAC: Authorizing `identity:get_project(project_id=1f7bba182b7349baa3f83745b9914b49)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:27.923847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9a821-91da-46b4-9fcc-cb1b623c60de None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:27.925502 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 593/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:44:27 2022] GET /identity/v3/projects/1f7bba182b7349baa3f83745b9914b49 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:27.931415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e35446-061f-4b1c-95ae-b3a7f254f5bb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:27.973870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:28.027413 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e35446-061f-4b1c-95ae-b3a7f254f5bb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:28.060997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ab13f-536c-4aa6-89ff-542de546120f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:28.061635 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ab13f-536c-4aa6-89ff-542de546120f None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:28.061912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ab13f-536c-4aa6-89ff-542de546120f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:28.062185 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ab13f-536c-4aa6-89ff-542de546120f None admin] PATH_INFO: `/v3/projects/1f7bba182b7349baa3f83745b9914b49/users/df1178bd868546b1908cfc8888730953/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:28.064055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ab13f-536c-4aa6-89ff-542de546120f None admin] RBAC: Authorizing `identity:create_grant(project_id=1f7bba182b7349baa3f83745b9914b49, user_id=df1178bd868546b1908cfc8888730953, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:28.065501 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ab13f-536c-4aa6-89ff-542de546120f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:28.077683 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 599/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:44:27 2022] PUT /identity/v3/projects/1f7bba182b7349baa3f83745b9914b49/users/df1178bd868546b1908cfc8888730953/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:44:30.198926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a2b3b-e027-490c-ad03-686cb05b5aca None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:30.199489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a2b3b-e027-490c-ad03-686cb05b5aca None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:30.199489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a2b3b-e027-490c-ad03-686cb05b5aca None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:30.201464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 594/1193] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:44:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:44:30.206732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22707ec9-7753-4719-a1b4-4b5376c959e3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:30.207102 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22707ec9-7753-4719-a1b4-4b5376c959e3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:30.207272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22707ec9-7753-4719-a1b4-4b5376c959e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:30.225082 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-22707ec9-7753-4719-a1b4-4b5376c959e3 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:44:30.226262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-22707ec9-7753-4719-a1b4-4b5376c959e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:44:30.290863 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 600/1194] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:44:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:44:30.558787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a2b3b-e027-490c-ad03-686cb05b5aca None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:30.657182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a2b3b-e027-490c-ad03-686cb05b5aca None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:30.667363 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:30.691071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03542a26-ddc0-4b3d-b2e6-7c1cd3c03255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:30.691704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03542a26-ddc0-4b3d-b2e6-7c1cd3c03255 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:30.691982 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03542a26-ddc0-4b3d-b2e6-7c1cd3c03255 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:30.692281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03542a26-ddc0-4b3d-b2e6-7c1cd3c03255 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:30.702213 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03542a26-ddc0-4b3d-b2e6-7c1cd3c03255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:44:30.706927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-03542a26-ddc0-4b3d-b2e6-7c1cd3c03255 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:44:30.706927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:44:30.706927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:44:30.706927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:44:30.706927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:44:30.706927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:44:30.706927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:44:30.706927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:44:30.706927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:44:30.706927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:44:30.706927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:44:30.706927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:44:30.706927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:44:30.706927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:44:30.706927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:44:30.706927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:44:30.708012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:44:30.708012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:44:30.708012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:44:30.708012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:44:30.708012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:44:30.708012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:44:30.708012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:44:30.708012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:44:30.708012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:44:30.708012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:44:30.708012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:44:30.708012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:44:30.708012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:44:30.708012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:44:30.708012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:44:30.708012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:44:30.709054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:44:30.709054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:44:30.709054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:44:30.709054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:44:30.709054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:44:30.709054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:44:30.709054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:44:30.709054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:44:30.709054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:44:30.709054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:44:30.709054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:44:30.709054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:44:30.709054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 595/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:44:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:44:30.713960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22707ec9-7753-4719-a1b4-4b5376c959e3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:30.743570 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:30.810413 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22707ec9-7753-4719-a1b4-4b5376c959e3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:30.842722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02f469-87d8-478c-96b2-34f1c211bc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:30.843305 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02f469-87d8-478c-96b2-34f1c211bc63 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:30.843579 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02f469-87d8-478c-96b2-34f1c211bc63 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:30.843843 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02f469-87d8-478c-96b2-34f1c211bc63 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:30.844513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02f469-87d8-478c-96b2-34f1c211bc63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:44:30.844810 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02f469-87d8-478c-96b2-34f1c211bc63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:30.846145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02f469-87d8-478c-96b2-34f1c211bc63 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:30.853755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 601/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:44:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:30.859552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03542a26-ddc0-4b3d-b2e6-7c1cd3c03255 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:30.956537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03542a26-ddc0-4b3d-b2e6-7c1cd3c03255 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:30.988774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a46ef2-942e-4f48-abac-0ddf4b4d8dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:30.989377 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a46ef2-942e-4f48-abac-0ddf4b4d8dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:30.989653 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a46ef2-942e-4f48-abac-0ddf4b4d8dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:30.989918 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a46ef2-942e-4f48-abac-0ddf4b4d8dd4 None admin] PATH_INFO: `/v3/users/df1178bd868546b1908cfc8888730953` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:30.991000 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a46ef2-942e-4f48-abac-0ddf4b4d8dd4 None admin] RBAC: Authorizing `identity:get_user(user_id=df1178bd868546b1908cfc8888730953)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:30.992454 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a46ef2-942e-4f48-abac-0ddf4b4d8dd4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:30.993996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 596/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 7 07:44:30 2022] GET /identity/v3/users/df1178bd868546b1908cfc8888730953 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:31.000266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02f469-87d8-478c-96b2-34f1c211bc63 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:31.090815 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:31.097461 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02f469-87d8-478c-96b2-34f1c211bc63 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:31.129537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88402eb-f1dd-425e-a7fc-429fb84a36b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:31.130117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88402eb-f1dd-425e-a7fc-429fb84a36b9 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:31.130390 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88402eb-f1dd-425e-a7fc-429fb84a36b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:31.130657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88402eb-f1dd-425e-a7fc-429fb84a36b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:31.194147 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88402eb-f1dd-425e-a7fc-429fb84a36b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:31.195556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88402eb-f1dd-425e-a7fc-429fb84a36b9 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:31.286357 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:31.294795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 602/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:44:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:44:31.300674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a46ef2-942e-4f48-abac-0ddf4b4d8dd4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:31.340053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:31.397688 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a46ef2-942e-4f48-abac-0ddf4b4d8dd4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:31.430676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f3bb8-fa98-4456-a9d8-d671e9eb3149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:31.431312 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f3bb8-fa98-4456-a9d8-d671e9eb3149 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:31.431589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f3bb8-fa98-4456-a9d8-d671e9eb3149 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:31.431858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f3bb8-fa98-4456-a9d8-d671e9eb3149 None admin] PATH_INFO: `/v3/projects/1f7bba182b7349baa3f83745b9914b49` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:31.432993 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f3bb8-fa98-4456-a9d8-d671e9eb3149 None admin] RBAC: Authorizing `identity:get_project(project_id=1f7bba182b7349baa3f83745b9914b49)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:31.435088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f3bb8-fa98-4456-a9d8-d671e9eb3149 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:31.436736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 597/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 7 07:44:31 2022] GET /identity/v3/projects/1f7bba182b7349baa3f83745b9914b49 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:31.442812 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88402eb-f1dd-425e-a7fc-429fb84a36b9 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:44:31.542228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88402eb-f1dd-425e-a7fc-429fb84a36b9 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:44:31.576020 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b147839-4259-4d5c-982a-98dae3db18e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:44:31.576635 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b147839-4259-4d5c-982a-98dae3db18e2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:44:31.576908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b147839-4259-4d5c-982a-98dae3db18e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:44:31.577174 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b147839-4259-4d5c-982a-98dae3db18e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:44:31.577881 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b147839-4259-4d5c-982a-98dae3db18e2 None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=1f7bba182b7349baa3f83745b9914b49, user.id=df1178bd868546b1908cfc8888730953) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:44:31.578185 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b147839-4259-4d5c-982a-98dae3db18e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:44:31.579616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b147839-4259-4d5c-982a-98dae3db18e2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:44:31.585129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 603/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:44:31 2022] GET /identity/v3/role_assignments?user.id=df1178bd868546b1908cfc8888730953&scope.project.id=1f7bba182b7349baa3f83745b9914b49&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:44:41.426592 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=47 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:44:41.582613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=71 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:27.521941 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114912a0-797d-4b04-a223-0377a4755f2a None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:27.522563 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114912a0-797d-4b04-a223-0377a4755f2a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:27.522563 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114912a0-797d-4b04-a223-0377a4755f2a None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:27.523828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 598/1201] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:45:27.529742 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cecf4-dfaf-49e6-aac2-761e0c8cc8c9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:27.530218 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cecf4-dfaf-49e6-aac2-761e0c8cc8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:27.530338 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cecf4-dfaf-49e6-aac2-761e0c8cc8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:27.549594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e16cecf4-dfaf-49e6-aac2-761e0c8cc8c9 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:45:27.550800 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e16cecf4-dfaf-49e6-aac2-761e0c8cc8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:45:27.628913 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 604/1202] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:45:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:45:27.914015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114912a0-797d-4b04-a223-0377a4755f2a None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:28.055737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114912a0-797d-4b04-a223-0377a4755f2a None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:28.101704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fa9c0-0b26-4132-b662-113954d11003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:28.102423 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fa9c0-0b26-4132-b662-113954d11003 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:28.102709 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fa9c0-0b26-4132-b662-113954d11003 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:28.102985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fa9c0-0b26-4132-b662-113954d11003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:28.116464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fa9c0-0b26-4132-b662-113954d11003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:28.118216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fa9c0-0b26-4132-b662-113954d11003 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:28.139436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-da7fa9c0-0b26-4132-b662-113954d11003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:28.139436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:28.139436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:28.139436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:28.139436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:28.139436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:28.139436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:28.139436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:28.139436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:28.139436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:28.139436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:28.139436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:28.139436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:45:28.139436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:45:28.139436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:45:28.139436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:28.140856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:45:28.141811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:45:28.141811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:28.141811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:45:28.145821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 599/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:28.147013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16cecf4-dfaf-49e6-aac2-761e0c8cc8c9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:28.270997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16cecf4-dfaf-49e6-aac2-761e0c8cc8c9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:28.302570 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:28.313752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1b731-12cc-46ab-ade7-1bbb50edd22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:28.314363 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1b731-12cc-46ab-ade7-1bbb50edd22f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:28.314718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1b731-12cc-46ab-ade7-1bbb50edd22f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:28.314910 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1b731-12cc-46ab-ade7-1bbb50edd22f None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:28.315585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1b731-12cc-46ab-ade7-1bbb50edd22f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:28.315866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1b731-12cc-46ab-ade7-1bbb50edd22f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:28.317256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1b731-12cc-46ab-ade7-1bbb50edd22f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:28.329510 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 605/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:45:28.335273 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fa9c0-0b26-4132-b662-113954d11003 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:28.396036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:28.463338 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fa9c0-0b26-4132-b662-113954d11003 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:28.511307 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afb823-c6c2-4302-82e4-a24aaed6f99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:28.512026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afb823-c6c2-4302-82e4-a24aaed6f99a None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:28.512281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afb823-c6c2-4302-82e4-a24aaed6f99a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:28.512498 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afb823-c6c2-4302-82e4-a24aaed6f99a None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:28.513138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afb823-c6c2-4302-82e4-a24aaed6f99a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:28.515353 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afb823-c6c2-4302-82e4-a24aaed6f99a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:28.538513 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 600/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 7 07:45:28 2022] POST /identity/v3/users => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:45:31.187831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0308393-b193-4cad-9cb8-a79bfe82d286 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:31.188397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0308393-b193-4cad-9cb8-a79bfe82d286 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:31.188397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0308393-b193-4cad-9cb8-a79bfe82d286 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:31.193071 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 606/1206] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:45:31.199170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278a15c-b7b4-4070-8303-fb6662c025a7 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:31.200114 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278a15c-b7b4-4070-8303-fb6662c025a7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:31.200114 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278a15c-b7b4-4070-8303-fb6662c025a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:31.221091 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-4278a15c-b7b4-4070-8303-fb6662c025a7 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:45:31.222774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4278a15c-b7b4-4070-8303-fb6662c025a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:45:31.320167 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 601/1207] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:45:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:45:31.684650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0308393-b193-4cad-9cb8-a79bfe82d286 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:31.828577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0308393-b193-4cad-9cb8-a79bfe82d286 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:31.866735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c2f93-47af-4786-96cf-01a596278a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:31.867417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c2f93-47af-4786-96cf-01a596278a08 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:31.867700 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c2f93-47af-4786-96cf-01a596278a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:31.867970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c2f93-47af-4786-96cf-01a596278a08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:31.879314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c2f93-47af-4786-96cf-01a596278a08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:45:31.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-872c2f93-47af-4786-96cf-01a596278a08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:45:31.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:31.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:31.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:31.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:31.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:31.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:31.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:31.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:31.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:31.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:31.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:31.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:45:31.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:45:31.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:45:31.884490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:45:31.889098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:45:31.889098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:45:31.889098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:45:31.889098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:45:31.889098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:31.889098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:31.889098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:31.889098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:31.889098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:31.889098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:31.889098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:31.889098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:31.889098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:31.889098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:31.889098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:31.889098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:31.890191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:31.890191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:31.890191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:31.890191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:31.890191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:45:31.890191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:45:31.890191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:45:31.890191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:45:31.890191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:45:31.890191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:45:31.890191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:45:31.890191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:45:31.890191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 607/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:45:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:31.892772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278a15c-b7b4-4070-8303-fb6662c025a7 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:31.940145 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:32.021623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278a15c-b7b4-4070-8303-fb6662c025a7 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:32.066662 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a89532-30d0-4f5b-9037-a262a27ae06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:32.067415 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a89532-30d0-4f5b-9037-a262a27ae06b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:32.067791 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a89532-30d0-4f5b-9037-a262a27ae06b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:32.069866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a89532-30d0-4f5b-9037-a262a27ae06b None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:32.069866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a89532-30d0-4f5b-9037-a262a27ae06b None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:32.069866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a89532-30d0-4f5b-9037-a262a27ae06b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:32.070788 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a89532-30d0-4f5b-9037-a262a27ae06b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:32.081811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 602/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:45:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:32.090436 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c2f93-47af-4786-96cf-01a596278a08 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:32.152642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:32.208650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c2f93-47af-4786-96cf-01a596278a08 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:32.248350 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a39571-93c5-4625-b13b-9831dc7edf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:32.249007 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a39571-93c5-4625-b13b-9831dc7edf09 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:32.249273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a39571-93c5-4625-b13b-9831dc7edf09 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:32.249541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a39571-93c5-4625-b13b-9831dc7edf09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:32.262406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a39571-93c5-4625-b13b-9831dc7edf09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:32.263732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a39571-93c5-4625-b13b-9831dc7edf09 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:32.279965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-70a39571-93c5-4625-b13b-9831dc7edf09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:32.279965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:32.279965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:32.279965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:32.279965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:32.279965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:32.279965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:32.279965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:32.279965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:32.279965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:32.279965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:32.279965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:32.279965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:45:32.279965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:45:32.279965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:45:32.279965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:32.287263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:45:32.288596 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:45:32.288596 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:32.288596 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:45:32.288596 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 608/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:32.288876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a89532-30d0-4f5b-9037-a262a27ae06b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:32.403201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:32.406534 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a89532-30d0-4f5b-9037-a262a27ae06b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:32.444705 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f772852-5063-4826-9257-e240ca5dbf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:32.445302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f772852-5063-4826-9257-e240ca5dbf9b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:32.445616 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f772852-5063-4826-9257-e240ca5dbf9b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:32.445939 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f772852-5063-4826-9257-e240ca5dbf9b None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:32.446612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f772852-5063-4826-9257-e240ca5dbf9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:32.446924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f772852-5063-4826-9257-e240ca5dbf9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:32.448591 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f772852-5063-4826-9257-e240ca5dbf9b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:32.468384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 603/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:32.469263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a39571-93c5-4625-b13b-9831dc7edf09 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:32.591494 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a39571-93c5-4625-b13b-9831dc7edf09 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:32.593078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:32.632053 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f24f4-7619-4c10-b669-a26bbb821fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:32.632713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f24f4-7619-4c10-b669-a26bbb821fca None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:32.632989 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f24f4-7619-4c10-b669-a26bbb821fca None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:32.633258 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f24f4-7619-4c10-b669-a26bbb821fca None admin] PATH_INFO: `/v3/users/placement` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:32.641793 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f24f4-7619-4c10-b669-a26bbb821fca None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:32.643514 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f24f4-7619-4c10-b669-a26bbb821fca None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:32.657597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-6f2f24f4-7619-4c10-b669-a26bbb821fca None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 07:45:32.657597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:32.657597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:32.657597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:32.657597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:32.657597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:32.657597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:32.657597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:32.657597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:32.657597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:32.657597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:32.657597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:32.657597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:45:32.657597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:45:32.657597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:45:32.657597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:32.658636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:32.659597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:32.659597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:32.659597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:45:32.659597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:45:32.659597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:45:32.659597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:45:32.659597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:45:32.659597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:45:32.659597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 07:45:32.659597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:45:32.663799 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 609/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:32.665193 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f772852-5063-4826-9257-e240ca5dbf9b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:32.791484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f772852-5063-4826-9257-e240ca5dbf9b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:32.836219 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50987c5-1927-411b-8b39-51eee45b8f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:32.836945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50987c5-1927-411b-8b39-51eee45b8f59 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:32.837321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50987c5-1927-411b-8b39-51eee45b8f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:32.837695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50987c5-1927-411b-8b39-51eee45b8f59 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:32.839142 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50987c5-1927-411b-8b39-51eee45b8f59 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:32.839558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50987c5-1927-411b-8b39-51eee45b8f59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:32.841201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50987c5-1927-411b-8b39-51eee45b8f59 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:32.867449 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 604/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Sep 7 07:45:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:32.868735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f24f4-7619-4c10-b669-a26bbb821fca None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:33.005167 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f24f4-7619-4c10-b669-a26bbb821fca None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:33.050632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe49d4-b8c2-422b-b91c-7d70e1d6b676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:33.051341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe49d4-b8c2-422b-b91c-7d70e1d6b676 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:33.051712 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe49d4-b8c2-422b-b91c-7d70e1d6b676 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:33.052107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe49d4-b8c2-422b-b91c-7d70e1d6b676 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:33.087678 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:33.131143 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe49d4-b8c2-422b-b91c-7d70e1d6b676 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:33.132727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe49d4-b8c2-422b-b91c-7d70e1d6b676 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:33.255107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 610/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:45:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:45:33.261010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50987c5-1927-411b-8b39-51eee45b8f59 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:33.285026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:33.386055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50987c5-1927-411b-8b39-51eee45b8f59 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:33.426990 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1201d4b-c200-4d95-aa58-c48953d0077b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:33.428018 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1201d4b-c200-4d95-aa58-c48953d0077b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:33.428536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1201d4b-c200-4d95-aa58-c48953d0077b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:33.429031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1201d4b-c200-4d95-aa58-c48953d0077b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:33.440729 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1201d4b-c200-4d95-aa58-c48953d0077b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:33.442933 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1201d4b-c200-4d95-aa58-c48953d0077b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:33.460367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-b1201d4b-c200-4d95-aa58-c48953d0077b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:33.460367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:33.460367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:33.460367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:33.460367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:33.460367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:33.460367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:33.460367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:33.460367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:33.460367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:33.460367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:33.460367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:33.460367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:45:33.460367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:45:33.460367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:45:33.460367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:33.461365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:45:33.462271 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:45:33.462271 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:33.462271 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:45:33.464735 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 605/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:33.470170 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe49d4-b8c2-422b-b91c-7d70e1d6b676 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:33.544950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:33.593454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe49d4-b8c2-422b-b91c-7d70e1d6b676 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:33.634108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa647f-cf2b-4f0e-bcd7-b6ebcbdbcb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:33.635389 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa647f-cf2b-4f0e-bcd7-b6ebcbdbcb2b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:33.635665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa647f-cf2b-4f0e-bcd7-b6ebcbdbcb2b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:33.636144 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa647f-cf2b-4f0e-bcd7-b6ebcbdbcb2b None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:33.636805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa647f-cf2b-4f0e-bcd7-b6ebcbdbcb2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:33.637118 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa647f-cf2b-4f0e-bcd7-b6ebcbdbcb2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:33.638496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa647f-cf2b-4f0e-bcd7-b6ebcbdbcb2b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:33.656034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 611/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:33.657420 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1201d4b-c200-4d95-aa58-c48953d0077b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:33.741003 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:33.781753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1201d4b-c200-4d95-aa58-c48953d0077b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:33.824321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a76369-3fd3-4b98-8206-1cf95f5b2524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:33.824958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a76369-3fd3-4b98-8206-1cf95f5b2524 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:33.825247 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a76369-3fd3-4b98-8206-1cf95f5b2524 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:33.825512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a76369-3fd3-4b98-8206-1cf95f5b2524 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:33.833538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a76369-3fd3-4b98-8206-1cf95f5b2524 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:33.836389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a76369-3fd3-4b98-8206-1cf95f5b2524 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:33.849945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-c2a76369-3fd3-4b98-8206-1cf95f5b2524 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:45:33.849945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:33.849945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:33.849945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:33.849945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:33.849945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:33.849945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:33.849945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:33.849945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:33.849945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:33.849945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:33.849945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:33.849945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:45:33.849945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:45:33.849945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:33.851199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:33.852285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:45:33.852285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:45:33.852285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:45:33.852285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:45:33.852285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:45:33.852285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:45:33.852285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:45:33.852285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:45:33.852285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 606/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:45:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:33.857137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa647f-cf2b-4f0e-bcd7-b6ebcbdbcb2b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:33.991592 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa647f-cf2b-4f0e-bcd7-b6ebcbdbcb2b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:34.026005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:34.037763 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2b01c-4b20-411e-90ea-b5b1c9fce54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:34.038391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2b01c-4b20-411e-90ea-b5b1c9fce54f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:34.038671 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2b01c-4b20-411e-90ea-b5b1c9fce54f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:34.038944 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2b01c-4b20-411e-90ea-b5b1c9fce54f None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:34.039632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2b01c-4b20-411e-90ea-b5b1c9fce54f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:34.039940 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2b01c-4b20-411e-90ea-b5b1c9fce54f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:34.041436 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2b01c-4b20-411e-90ea-b5b1c9fce54f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:34.054211 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 612/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:45:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:45:34.060889 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a76369-3fd3-4b98-8206-1cf95f5b2524 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:34.173562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a76369-3fd3-4b98-8206-1cf95f5b2524 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:34.205480 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:34.212410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536baf7-85e0-454e-8f35-c0978f1f37d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:34.213089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536baf7-85e0-454e-8f35-c0978f1f37d8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:34.213365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536baf7-85e0-454e-8f35-c0978f1f37d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:34.213857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536baf7-85e0-454e-8f35-c0978f1f37d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:34.214885 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536baf7-85e0-454e-8f35-c0978f1f37d8 None admin] RBAC: Adding query filter params (role.id=e7960e6d957f4a1fa6547565db026f24, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=b16b81ab30294bc4a8c87b6d7c3573f0) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:34.215366 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536baf7-85e0-454e-8f35-c0978f1f37d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:34.216852 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536baf7-85e0-454e-8f35-c0978f1f37d8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:34.224252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 607/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:45:34 2022] GET /identity/v3/role_assignments?user.id=b16b81ab30294bc4a8c87b6d7c3573f0&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=e7960e6d957f4a1fa6547565db026f24 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:36.480687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6f650-ca67-4c76-9677-cc7fa2600dbe None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:36.481275 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6f650-ca67-4c76-9677-cc7fa2600dbe None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:36.481389 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6f650-ca67-4c76-9677-cc7fa2600dbe None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:36.482805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 613/1220] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:45:36.488794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea434c-ac4a-4823-9351-884caf7ed0b0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:36.489240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea434c-ac4a-4823-9351-884caf7ed0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:36.489338 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea434c-ac4a-4823-9351-884caf7ed0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:36.507611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-77ea434c-ac4a-4823-9351-884caf7ed0b0 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:45:36.508831 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-77ea434c-ac4a-4823-9351-884caf7ed0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:45:36.585466 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 608/1221] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:45:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:45:36.901646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6f650-ca67-4c76-9677-cc7fa2600dbe None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:37.035132 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6f650-ca67-4c76-9677-cc7fa2600dbe None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:37.079900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83205a0-8abc-4f6e-8881-83ff367470e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:37.080680 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83205a0-8abc-4f6e-8881-83ff367470e7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:37.081070 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83205a0-8abc-4f6e-8881-83ff367470e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:37.081464 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83205a0-8abc-4f6e-8881-83ff367470e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:37.092163 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83205a0-8abc-4f6e-8881-83ff367470e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:45:37.097241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f83205a0-8abc-4f6e-8881-83ff367470e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:45:37.097241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:37.097241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:37.097241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:37.097241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:37.097241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:37.097241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:37.097241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:37.097241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:37.097241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:37.097241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:37.097241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:37.097241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:45:37.097241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:45:37.097241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:45:37.097241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:45:37.098245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:45:37.098245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:45:37.098245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:45:37.098245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:45:37.098245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:37.098245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:37.098245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:37.098245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:37.098245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:37.098245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:37.098245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:37.098245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:37.098245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:37.098245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:37.098245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:37.098245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:37.099072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:37.099072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:37.099072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:37.099072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:37.099072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:45:37.099072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:45:37.099072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:45:37.099072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:45:37.099072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:45:37.099072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:45:37.099072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:45:37.099072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:45:37.099072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 614/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:45:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:37.104472 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea434c-ac4a-4823-9351-884caf7ed0b0 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:37.217492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea434c-ac4a-4823-9351-884caf7ed0b0 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:37.247123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:37.259056 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ee134-ffd9-47e5-a4ed-fdbb28903b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:37.260286 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ee134-ffd9-47e5-a4ed-fdbb28903b24 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:37.260796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ee134-ffd9-47e5-a4ed-fdbb28903b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:37.261108 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ee134-ffd9-47e5-a4ed-fdbb28903b24 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:37.261775 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ee134-ffd9-47e5-a4ed-fdbb28903b24 None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:37.262078 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ee134-ffd9-47e5-a4ed-fdbb28903b24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:37.263774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ee134-ffd9-47e5-a4ed-fdbb28903b24 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:37.273185 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 609/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:45:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:37.279564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83205a0-8abc-4f6e-8881-83ff367470e7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:37.329177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:37.393938 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83205a0-8abc-4f6e-8881-83ff367470e7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:37.435649 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7056c3d-5329-4e3e-8073-415a3e1da6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:37.436322 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7056c3d-5329-4e3e-8073-415a3e1da6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:37.436585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7056c3d-5329-4e3e-8073-415a3e1da6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:37.436855 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7056c3d-5329-4e3e-8073-415a3e1da6c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:37.448752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7056c3d-5329-4e3e-8073-415a3e1da6c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:37.450188 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7056c3d-5329-4e3e-8073-415a3e1da6c5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:37.465154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-e7056c3d-5329-4e3e-8073-415a3e1da6c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:37.465154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:37.465154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:37.465154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:37.465154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:37.465154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:37.465154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:37.465154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:37.465154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:37.465154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:37.465154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:37.465154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:37.465154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:45:37.465154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:45:37.465154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:45:37.465154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:37.466145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:45:37.467035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:45:37.467035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:37.467035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:45:37.467035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 615/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:37.472219 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ee134-ffd9-47e5-a4ed-fdbb28903b24 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:37.577406 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ee134-ffd9-47e5-a4ed-fdbb28903b24 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:37.615005 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b860e-da4f-4972-b846-2ff4fa15096b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:37.615610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b860e-da4f-4972-b846-2ff4fa15096b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:37.615889 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b860e-da4f-4972-b846-2ff4fa15096b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:37.616176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b860e-da4f-4972-b846-2ff4fa15096b None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:37.616793 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b860e-da4f-4972-b846-2ff4fa15096b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:37.617090 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b860e-da4f-4972-b846-2ff4fa15096b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:37.618449 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b860e-da4f-4972-b846-2ff4fa15096b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:37.631311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 610/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:37.637844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7056c3d-5329-4e3e-8073-415a3e1da6c5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:37.746235 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:37.753759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7056c3d-5329-4e3e-8073-415a3e1da6c5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:37.788912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ced2e0-4ef8-447b-8bf4-001972ac1da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:37.789534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ced2e0-4ef8-447b-8bf4-001972ac1da0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:37.789805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ced2e0-4ef8-447b-8bf4-001972ac1da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:37.790068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ced2e0-4ef8-447b-8bf4-001972ac1da0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:37.797724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ced2e0-4ef8-447b-8bf4-001972ac1da0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:37.799136 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ced2e0-4ef8-447b-8bf4-001972ac1da0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:37.812476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-71ced2e0-4ef8-447b-8bf4-001972ac1da0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 07:45:37.812476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:37.812476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:37.812476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:37.812476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:37.812476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:37.812476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:37.812476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:37.812476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:37.812476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:37.812476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:37.812476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:37.812476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:45:37.812476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:45:37.812476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:45:37.812476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:37.813685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:37.814893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:37.814893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:37.814893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:45:37.814893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:45:37.814893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:45:37.814893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:45:37.814893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:45:37.814893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:45:37.814893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 07:45:37.814893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:45:37.814893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 616/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:37.819867 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b860e-da4f-4972-b846-2ff4fa15096b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:37.875530 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:37.954607 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b860e-da4f-4972-b846-2ff4fa15096b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:37.995135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fff74-5143-4121-92bc-7c78b304650b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:37.995740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fff74-5143-4121-92bc-7c78b304650b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:37.996014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fff74-5143-4121-92bc-7c78b304650b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:37.996354 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fff74-5143-4121-92bc-7c78b304650b None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:37.997028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fff74-5143-4121-92bc-7c78b304650b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:37.997334 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fff74-5143-4121-92bc-7c78b304650b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:37.998743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fff74-5143-4121-92bc-7c78b304650b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:38.015490 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 611/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Sep 7 07:45:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:38.021292 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ced2e0-4ef8-447b-8bf4-001972ac1da0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:38.126007 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ced2e0-4ef8-447b-8bf4-001972ac1da0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:38.160509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d45d35-3713-48cd-b325-13dc6d7c8f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:38.161129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d45d35-3713-48cd-b325-13dc6d7c8f13 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:38.161404 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d45d35-3713-48cd-b325-13dc6d7c8f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:38.161673 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d45d35-3713-48cd-b325-13dc6d7c8f13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:38.172363 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d45d35-3713-48cd-b325-13dc6d7c8f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:38.173788 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d45d35-3713-48cd-b325-13dc6d7c8f13 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:38.187953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-b3d45d35-3713-48cd-b325-13dc6d7c8f13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:38.187953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:38.187953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:38.187953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:38.187953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:38.187953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:38.187953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:38.187953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:38.187953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:38.187953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:38.187953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:38.187953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:38.187953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:45:38.187953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:45:38.187953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:45:38.187953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:38.194067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:45:38.195173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:45:38.195173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:38.195173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:45:38.195173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 617/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:38.196399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fff74-5143-4121-92bc-7c78b304650b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:38.315103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fff74-5143-4121-92bc-7c78b304650b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:38.318348 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:38.360046 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569d788-5354-4853-a43c-22375f81ac88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:38.360679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569d788-5354-4853-a43c-22375f81ac88 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:38.360982 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569d788-5354-4853-a43c-22375f81ac88 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:38.361277 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569d788-5354-4853-a43c-22375f81ac88 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:38.361977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569d788-5354-4853-a43c-22375f81ac88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:38.362279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569d788-5354-4853-a43c-22375f81ac88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:38.363833 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569d788-5354-4853-a43c-22375f81ac88 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:38.377072 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 612/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:38.382690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d45d35-3713-48cd-b325-13dc6d7c8f13 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:38.392890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:38.488518 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d45d35-3713-48cd-b325-13dc6d7c8f13 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:38.524941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91844c-8f9e-4445-9151-1bee7424405a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:38.525693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91844c-8f9e-4445-9151-1bee7424405a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:38.526055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91844c-8f9e-4445-9151-1bee7424405a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:38.526282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91844c-8f9e-4445-9151-1bee7424405a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:38.532920 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91844c-8f9e-4445-9151-1bee7424405a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:38.534373 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91844c-8f9e-4445-9151-1bee7424405a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:38.545603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ef91844c-8f9e-4445-9151-1bee7424405a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:45:38.545603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:38.545603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:38.545603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:38.545603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:38.545603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:38.545603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:38.545603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:38.545603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:38.545603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:38.545603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:38.545603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:38.545603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:45:38.545603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:45:38.545603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:38.547026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:38.548234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:45:38.548234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:45:38.548234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:45:38.548234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:45:38.548234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:45:38.548234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:45:38.548234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:45:38.548234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:45:38.548234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 618/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:45:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:38.552707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569d788-5354-4853-a43c-22375f81ac88 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:38.662434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569d788-5354-4853-a43c-22375f81ac88 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:38.699420 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8564127-60f7-472f-b430-46f1be48e4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:38.700069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8564127-60f7-472f-b430-46f1be48e4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:38.700349 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8564127-60f7-472f-b430-46f1be48e4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:38.700624 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8564127-60f7-472f-b430-46f1be48e4d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:38.701331 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8564127-60f7-472f-b430-46f1be48e4d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:38.701638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8564127-60f7-472f-b430-46f1be48e4d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:38.703094 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8564127-60f7-472f-b430-46f1be48e4d7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:38.715930 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 613/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:45:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:45:38.722052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91844c-8f9e-4445-9151-1bee7424405a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:38.803989 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:38.842948 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91844c-8f9e-4445-9151-1bee7424405a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:38.877535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e699d3f-513a-4df6-a9a4-fbc117f2b722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:38.878183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e699d3f-513a-4df6-a9a4-fbc117f2b722 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:38.878455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e699d3f-513a-4df6-a9a4-fbc117f2b722 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:38.878722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e699d3f-513a-4df6-a9a4-fbc117f2b722 None admin] PATH_INFO: `/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/b16b81ab30294bc4a8c87b6d7c3573f0/roles/e7960e6d957f4a1fa6547565db026f24` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:38.893230 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e699d3f-513a-4df6-a9a4-fbc117f2b722 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a949b959e584474aafb4a05ebbabcc4, user_id=b16b81ab30294bc4a8c87b6d7c3573f0, role_id=e7960e6d957f4a1fa6547565db026f24)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:38.894651 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e699d3f-513a-4df6-a9a4-fbc117f2b722 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:38.906876 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 619/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:45:38 2022] PUT /identity/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/b16b81ab30294bc4a8c87b6d7c3573f0/roles/e7960e6d957f4a1fa6547565db026f24 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:45:41.314187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53fcc1-dbce-4294-9715-645851192947 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:41.314772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53fcc1-dbce-4294-9715-645851192947 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:41.314772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53fcc1-dbce-4294-9715-645851192947 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:41.316505 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 614/1233] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:45:41.325851 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16099ae-ba46-4e79-bdb3-e6dce20197e5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:41.325851 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16099ae-ba46-4e79-bdb3-e6dce20197e5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:41.325851 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16099ae-ba46-4e79-bdb3-e6dce20197e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:41.342864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d16099ae-ba46-4e79-bdb3-e6dce20197e5 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:45:41.344213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d16099ae-ba46-4e79-bdb3-e6dce20197e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:45:41.418527 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 620/1234] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:45:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:45:41.733492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53fcc1-dbce-4294-9715-645851192947 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:41.761844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:41.851485 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53fcc1-dbce-4294-9715-645851192947 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:41.888115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8692c61-973c-460f-a4c6-1cfb6522663e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:41.889044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8692c61-973c-460f-a4c6-1cfb6522663e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:41.889316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8692c61-973c-460f-a4c6-1cfb6522663e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:41.889565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8692c61-973c-460f-a4c6-1cfb6522663e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:41.899793 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8692c61-973c-460f-a4c6-1cfb6522663e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:45:41.905416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-e8692c61-973c-460f-a4c6-1cfb6522663e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:45:41.905416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:41.905416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:41.905416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:41.905416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:41.905416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:41.905416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:41.905416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:41.905416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:41.905416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:41.905416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:41.905416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:41.905416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:45:41.905416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:45:41.905416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:45:41.905416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:45:41.906374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:45:41.906374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:45:41.906374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:45:41.906374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:45:41.906374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:41.906374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:41.906374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:41.906374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:41.906374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:41.906374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:41.906374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:41.906374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:41.906374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:41.906374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:41.906374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:41.906374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:41.907220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:41.907220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:41.907220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:41.907220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:41.907220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:45:41.907220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:45:41.907220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:45:41.907220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:45:41.907220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:45:41.907220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:45:41.907220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 07:45:41.907220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:45:41.907220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 615/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 7 07:45:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:41.912480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16099ae-ba46-4e79-bdb3-e6dce20197e5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:41.960582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:42.018910 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16099ae-ba46-4e79-bdb3-e6dce20197e5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:42.054825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdc27f-5bf6-4cf0-838c-c29a22f8fa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:42.055428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdc27f-5bf6-4cf0-838c-c29a22f8fa8a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:42.055701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdc27f-5bf6-4cf0-838c-c29a22f8fa8a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:42.056038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdc27f-5bf6-4cf0-838c-c29a22f8fa8a None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:42.057343 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cdc27f-5bf6-4cf0-838c-c29a22f8fa8a None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:42.057781 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cdc27f-5bf6-4cf0-838c-c29a22f8fa8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:42.059130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cdc27f-5bf6-4cf0-838c-c29a22f8fa8a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:42.067406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 621/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 7 07:45:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:42.074210 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8692c61-973c-460f-a4c6-1cfb6522663e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:42.162360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:42.179293 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8692c61-973c-460f-a4c6-1cfb6522663e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:42.218160 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ee6f5-54c8-4a42-b6d1-548e6985849c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:42.218856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ee6f5-54c8-4a42-b6d1-548e6985849c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:42.219138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ee6f5-54c8-4a42-b6d1-548e6985849c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:42.219414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ee6f5-54c8-4a42-b6d1-548e6985849c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:42.231158 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ee6f5-54c8-4a42-b6d1-548e6985849c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:42.232655 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ee6f5-54c8-4a42-b6d1-548e6985849c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:42.248439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-3b0ee6f5-54c8-4a42-b6d1-548e6985849c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:42.248439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:42.248439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:42.248439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:42.248439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:42.248439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:42.248439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:42.248439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:42.248439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:42.248439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:42.248439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:42.248439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:42.248439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:45:42.248439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:45:42.248439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:45:42.248439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:42.249834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:45:42.251031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:45:42.251031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:42.251031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:45:42.251031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 616/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:45:42.255287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdc27f-5bf6-4cf0-838c-c29a22f8fa8a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:42.362278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdc27f-5bf6-4cf0-838c-c29a22f8fa8a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:42.372065 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:42.399826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab730c-8879-46f5-83f3-df58289db531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:42.401085 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab730c-8879-46f5-83f3-df58289db531 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:42.401361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab730c-8879-46f5-83f3-df58289db531 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:42.401632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab730c-8879-46f5-83f3-df58289db531 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:42.402269 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab730c-8879-46f5-83f3-df58289db531 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:42.402572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab730c-8879-46f5-83f3-df58289db531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:42.403978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab730c-8879-46f5-83f3-df58289db531 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:42.417217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 622/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:42.422133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ee6f5-54c8-4a42-b6d1-548e6985849c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:42.532684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ee6f5-54c8-4a42-b6d1-548e6985849c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:42.571616 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f385555-2b70-4ed2-a22f-83e4d121f6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:42.572845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f385555-2b70-4ed2-a22f-83e4d121f6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:42.573119 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f385555-2b70-4ed2-a22f-83e4d121f6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:42.573452 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f385555-2b70-4ed2-a22f-83e4d121f6a0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:42.576393 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:42.582331 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f385555-2b70-4ed2-a22f-83e4d121f6a0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:42.583936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f385555-2b70-4ed2-a22f-83e4d121f6a0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:42.596028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-3f385555-2b70-4ed2-a22f-83e4d121f6a0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 07:45:42.596028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:42.596028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:42.596028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:42.596028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:42.596028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:42.596028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:42.596028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:42.596028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:42.596028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:42.596028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:42.596028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:42.596028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:45:42.596028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:45:42.596028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:45:42.596028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:42.597237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:42.598520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:42.598520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:42.598520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:45:42.598520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:45:42.598520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:45:42.598520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:45:42.598520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:45:42.598520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:45:42.598520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 07:45:42.598520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:45:42.598520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 617/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:42.603717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab730c-8879-46f5-83f3-df58289db531 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:42.705575 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab730c-8879-46f5-83f3-df58289db531 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:42.743500 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e76884-4f74-4e74-9b62-c96175736acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:42.744554 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e76884-4f74-4e74-9b62-c96175736acc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:42.744861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e76884-4f74-4e74-9b62-c96175736acc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:42.745169 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e76884-4f74-4e74-9b62-c96175736acc None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:42.745958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e76884-4f74-4e74-9b62-c96175736acc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:42.746294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e76884-4f74-4e74-9b62-c96175736acc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:42.747739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e76884-4f74-4e74-9b62-c96175736acc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:42.768620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 623/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Sep 7 07:45:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:42.774205 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f385555-2b70-4ed2-a22f-83e4d121f6a0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:42.884585 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f385555-2b70-4ed2-a22f-83e4d121f6a0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:42.919307 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fc1d1-14cd-4e45-a559-1cdb1a22b2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:42.919915 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fc1d1-14cd-4e45-a559-1cdb1a22b2dd None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:42.920227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fc1d1-14cd-4e45-a559-1cdb1a22b2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:42.920495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fc1d1-14cd-4e45-a559-1cdb1a22b2dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:42.990902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:42.999662 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fc1d1-14cd-4e45-a559-1cdb1a22b2dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:43.001152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fc1d1-14cd-4e45-a559-1cdb1a22b2dd None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:43.119327 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 618/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:45:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:45:43.125221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e76884-4f74-4e74-9b62-c96175736acc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:43.154102 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:43.245880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e76884-4f74-4e74-9b62-c96175736acc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:43.285413 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114eab3-1b36-492b-8207-4874efb52d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:43.286238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114eab3-1b36-492b-8207-4874efb52d5a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:43.286439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114eab3-1b36-492b-8207-4874efb52d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:43.286810 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114eab3-1b36-492b-8207-4874efb52d5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:43.299749 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114eab3-1b36-492b-8207-4874efb52d5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:43.301721 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114eab3-1b36-492b-8207-4874efb52d5a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:43.319166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f114eab3-1b36-492b-8207-4874efb52d5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:43.319166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:43.319166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:43.319166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:43.319166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:43.319166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:43.319166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:43.319166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:43.319166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:43.319166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:43.319166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:43.319166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:43.319166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:45:43.319166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:45:43.319166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:45:43.319166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:43.320445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:45:43.321488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:45:43.321488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:43.321488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:45:43.321488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 624/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:45:43.326895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fc1d1-14cd-4e45-a559-1cdb1a22b2dd None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:43.439383 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:43.445645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fc1d1-14cd-4e45-a559-1cdb1a22b2dd None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:43.482384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c20374-2510-482f-ab76-7069dbe6ddc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:43.482993 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c20374-2510-482f-ab76-7069dbe6ddc7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:43.483267 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c20374-2510-482f-ab76-7069dbe6ddc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:43.483531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c20374-2510-482f-ab76-7069dbe6ddc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:43.484190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c20374-2510-482f-ab76-7069dbe6ddc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:43.484486 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c20374-2510-482f-ab76-7069dbe6ddc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:43.485979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c20374-2510-482f-ab76-7069dbe6ddc7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:43.499843 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 619/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:43.506449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114eab3-1b36-492b-8207-4874efb52d5a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:43.596967 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:43.625133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114eab3-1b36-492b-8207-4874efb52d5a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:43.661599 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e11e5-a37d-427f-bd67-05b0ff683fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:43.662223 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e11e5-a37d-427f-bd67-05b0ff683fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:43.662496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e11e5-a37d-427f-bd67-05b0ff683fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:43.662766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e11e5-a37d-427f-bd67-05b0ff683fd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:43.670140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e11e5-a37d-427f-bd67-05b0ff683fd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:43.671547 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e11e5-a37d-427f-bd67-05b0ff683fd7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:43.682769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ce7e11e5-a37d-427f-bd67-05b0ff683fd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:45:43.682769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:43.682769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:43.682769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:43.682769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:43.682769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:43.682769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:43.682769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:43.682769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:43.682769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:43.682769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:43.682769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:43.682769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:45:43.682769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:45:43.682769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:43.683679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:43.685052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:45:43.685052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:45:43.685052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:45:43.685052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:45:43.685052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:45:43.685052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:45:43.685052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:45:43.685052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:45:43.685052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 625/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:45:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 07:45:43.689800 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c20374-2510-482f-ab76-7069dbe6ddc7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:43.815721 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c20374-2510-482f-ab76-7069dbe6ddc7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:43.852299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b08da5-f76b-4213-b9c7-1e753a5ba8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:43.852902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b08da5-f76b-4213-b9c7-1e753a5ba8bc None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:43.853728 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b08da5-f76b-4213-b9c7-1e753a5ba8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:43.854002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b08da5-f76b-4213-b9c7-1e753a5ba8bc None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:43.854676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b08da5-f76b-4213-b9c7-1e753a5ba8bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:43.854977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b08da5-f76b-4213-b9c7-1e753a5ba8bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:43.856507 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b08da5-f76b-4213-b9c7-1e753a5ba8bc None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:43.868891 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 620/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:45:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:43.874992 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e11e5-a37d-427f-bd67-05b0ff683fd7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:43.981397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e11e5-a37d-427f-bd67-05b0ff683fd7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:44.018958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4e5d2-a951-45cc-bff7-833de7e3ee04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:44.019622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4e5d2-a951-45cc-bff7-833de7e3ee04 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:44.019928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4e5d2-a951-45cc-bff7-833de7e3ee04 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:44.020245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4e5d2-a951-45cc-bff7-833de7e3ee04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:44.020964 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4e5d2-a951-45cc-bff7-833de7e3ee04 None admin] RBAC: Adding query filter params (role.id=e7960e6d957f4a1fa6547565db026f24, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=b16b81ab30294bc4a8c87b6d7c3573f0) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:44.021272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4e5d2-a951-45cc-bff7-833de7e3ee04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:44.022681 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4e5d2-a951-45cc-bff7-833de7e3ee04 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:44.030283 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 626/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:45:43 2022] GET /identity/v3/role_assignments?user.id=b16b81ab30294bc4a8c87b6d7c3573f0&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=e7960e6d957f4a1fa6547565db026f24 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:45:46.503898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8654674-0fc9-42b3-9c36-b38f81c9289f None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:46.504506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8654674-0fc9-42b3-9c36-b38f81c9289f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:46.504506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8654674-0fc9-42b3-9c36-b38f81c9289f None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:46.505948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 621/1247] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:45:46.513665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ef992-923d-4304-bdcf-f479a65659d5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:46.514076 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ef992-923d-4304-bdcf-f479a65659d5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:46.514268 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ef992-923d-4304-bdcf-f479a65659d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:46.520459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:46.538005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-862ef992-923d-4304-bdcf-f479a65659d5 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:45:46.542985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-862ef992-923d-4304-bdcf-f479a65659d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:45:46.620945 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 627/1248] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:45:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:45:46.895875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8654674-0fc9-42b3-9c36-b38f81c9289f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:46.914994 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:47.016192 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8654674-0fc9-42b3-9c36-b38f81c9289f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:47.053727 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abb60f-7763-4dc1-914c-ccdcf979c3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:47.054351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abb60f-7763-4dc1-914c-ccdcf979c3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:47.054629 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abb60f-7763-4dc1-914c-ccdcf979c3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:47.054900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abb60f-7763-4dc1-914c-ccdcf979c3d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:47.066731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abb60f-7763-4dc1-914c-ccdcf979c3d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:45:47.072911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-99abb60f-7763-4dc1-914c-ccdcf979c3d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:45:47.072911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:47.072911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:47.072911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:47.072911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:47.072911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:47.072911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:47.072911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:47.072911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:47.072911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:47.072911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:47.072911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:47.072911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:45:47.072911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:45:47.072911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:45:47.072911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:45:47.074399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:45:47.074399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:45:47.074399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:45:47.074399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:45:47.074399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:47.074399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:47.074399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:47.074399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:47.074399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:47.074399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:47.074399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:47.074399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:47.074399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:47.074399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:47.074399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:47.074399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:47.075470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:47.075470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:47.075470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:47.075470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:47.075470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:45:47.075470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:45:47.075470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:45:47.075470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:45:47.075470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:45:47.075470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:45:47.075470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:45:47.075470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:45:47.075470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 622/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:45:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:47.080146 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ef992-923d-4304-bdcf-f479a65659d5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:47.197354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ef992-923d-4304-bdcf-f479a65659d5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:47.232319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:47.238141 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be8917-26b7-4e42-aa0d-06f705f9668a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:47.238975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be8917-26b7-4e42-aa0d-06f705f9668a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:47.239517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be8917-26b7-4e42-aa0d-06f705f9668a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:47.239804 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be8917-26b7-4e42-aa0d-06f705f9668a None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:47.240605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be8917-26b7-4e42-aa0d-06f705f9668a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:47.240919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be8917-26b7-4e42-aa0d-06f705f9668a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:47.242418 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be8917-26b7-4e42-aa0d-06f705f9668a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:47.252070 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 628/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:45:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:47.258189 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abb60f-7763-4dc1-914c-ccdcf979c3d9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:47.346212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:47.373203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abb60f-7763-4dc1-914c-ccdcf979c3d9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:47.412127 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3727fb-a3c0-4687-b91e-8c5baa2c5382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:47.412794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3727fb-a3c0-4687-b91e-8c5baa2c5382 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:47.416125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3727fb-a3c0-4687-b91e-8c5baa2c5382 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:47.416125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3727fb-a3c0-4687-b91e-8c5baa2c5382 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:47.426212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3727fb-a3c0-4687-b91e-8c5baa2c5382 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:47.427745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3727fb-a3c0-4687-b91e-8c5baa2c5382 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:47.445491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-5e3727fb-a3c0-4687-b91e-8c5baa2c5382 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:47.445491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:47.445491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:47.445491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:47.445491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:47.445491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:47.445491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:47.445491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:47.445491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:47.445491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:47.445491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:47.445491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:47.445491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:45:47.445491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:45:47.445491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:45:47.445491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:47.446747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:45:47.447932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:45:47.447932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:47.447932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:45:47.447932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 623/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:47.453307 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be8917-26b7-4e42-aa0d-06f705f9668a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:47.562774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be8917-26b7-4e42-aa0d-06f705f9668a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:47.598112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0685d-b508-433b-b925-7ffd2b964ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:47.598719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0685d-b508-433b-b925-7ffd2b964ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:47.598994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0685d-b508-433b-b925-7ffd2b964ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:47.599264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0685d-b508-433b-b925-7ffd2b964ef4 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:47.599906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0685d-b508-433b-b925-7ffd2b964ef4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:47.600240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0685d-b508-433b-b925-7ffd2b964ef4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:47.601608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0685d-b508-433b-b925-7ffd2b964ef4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:47.615194 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 629/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:47.620857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3727fb-a3c0-4687-b91e-8c5baa2c5382 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:47.726161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3727fb-a3c0-4687-b91e-8c5baa2c5382 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:47.755112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:47.763375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b534212-3513-4b2a-b3df-3634fde74d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:47.764014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b534212-3513-4b2a-b3df-3634fde74d00 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:47.764320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b534212-3513-4b2a-b3df-3634fde74d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:47.764588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b534212-3513-4b2a-b3df-3634fde74d00 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:47.772868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b534212-3513-4b2a-b3df-3634fde74d00 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:47.774404 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b534212-3513-4b2a-b3df-3634fde74d00 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:47.786537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-6b534212-3513-4b2a-b3df-3634fde74d00 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 07:45:47.786537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:47.786537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:47.786537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:47.786537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:47.786537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:47.786537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:47.786537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:47.786537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:47.786537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:47.786537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:47.786537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:47.786537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:45:47.786537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:45:47.786537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:45:47.786537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:47.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:47.788879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:47.788879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:47.788879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:45:47.788879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:45:47.788879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:45:47.788879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:45:47.788879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:45:47.788879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:45:47.788879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 07:45:47.788879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:45:47.788879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 624/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:47.794278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0685d-b508-433b-b925-7ffd2b964ef4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:47.852126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:47.897672 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0685d-b508-433b-b925-7ffd2b964ef4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:47.932577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116e453-63cb-424b-ad3c-f106842deb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:47.933178 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116e453-63cb-424b-ad3c-f106842deb89 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:47.933455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116e453-63cb-424b-ad3c-f106842deb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:47.933725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116e453-63cb-424b-ad3c-f106842deb89 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:47.934451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116e453-63cb-424b-ad3c-f106842deb89 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:47.934750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116e453-63cb-424b-ad3c-f106842deb89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:47.936204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116e453-63cb-424b-ad3c-f106842deb89 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:47.955317 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 630/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Sep 7 07:45:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:47.963761 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b534212-3513-4b2a-b3df-3634fde74d00 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:48.077026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b534212-3513-4b2a-b3df-3634fde74d00 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:48.111274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131da553-f780-45b9-947f-843de454b2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:48.111870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131da553-f780-45b9-947f-843de454b2ef None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:48.112179 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131da553-f780-45b9-947f-843de454b2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:48.112451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131da553-f780-45b9-947f-843de454b2ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:48.166082 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:48.178367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131da553-f780-45b9-947f-843de454b2ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:48.179803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131da553-f780-45b9-947f-843de454b2ef None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:48.273875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 625/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:45:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:45:48.280198 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116e453-63cb-424b-ad3c-f106842deb89 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:48.377022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116e453-63cb-424b-ad3c-f106842deb89 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:48.390849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:48.412489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d8b60-f844-4731-82d8-8554b189f4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:48.413116 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d8b60-f844-4731-82d8-8554b189f4bd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:48.413392 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d8b60-f844-4731-82d8-8554b189f4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:48.413663 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d8b60-f844-4731-82d8-8554b189f4bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:48.423519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d8b60-f844-4731-82d8-8554b189f4bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:48.424968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d8b60-f844-4731-82d8-8554b189f4bd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:48.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-550d8b60-f844-4731-82d8-8554b189f4bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:48.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:48.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:48.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:48.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:48.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:48.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:48.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:48.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:48.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:48.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:48.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:48.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:45:48.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:45:48.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:45:48.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:48.440238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:45:48.441403 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:45:48.441403 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:48.441403 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:45:48.441403 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 631/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:48.446129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131da553-f780-45b9-947f-843de454b2ef None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:48.531604 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:48.543629 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131da553-f780-45b9-947f-843de454b2ef None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:48.576742 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86d626-4680-4941-ad5a-6c6ee8aa79c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:48.577332 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86d626-4680-4941-ad5a-6c6ee8aa79c9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:48.577608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86d626-4680-4941-ad5a-6c6ee8aa79c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:48.577878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86d626-4680-4941-ad5a-6c6ee8aa79c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:48.578500 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86d626-4680-4941-ad5a-6c6ee8aa79c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:48.578800 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86d626-4680-4941-ad5a-6c6ee8aa79c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:48.580181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86d626-4680-4941-ad5a-6c6ee8aa79c9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:48.591600 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 626/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:48.597226 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d8b60-f844-4731-82d8-8554b189f4bd None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:48.693282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d8b60-f844-4731-82d8-8554b189f4bd None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:48.725926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2ebd2-d5c9-471c-96da-b39ba1f93eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:48.726521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2ebd2-d5c9-471c-96da-b39ba1f93eba None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:48.726796 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2ebd2-d5c9-471c-96da-b39ba1f93eba None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:48.727067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2ebd2-d5c9-471c-96da-b39ba1f93eba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:48.733408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2ebd2-d5c9-471c-96da-b39ba1f93eba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:48.734818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2ebd2-d5c9-471c-96da-b39ba1f93eba None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:48.746865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-89e2ebd2-d5c9-471c-96da-b39ba1f93eba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:45:48.746865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:48.746865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:48.746865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:48.746865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:48.746865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:48.746865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:48.746865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:48.746865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:48.746865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:48.746865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:48.746865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:48.746865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:45:48.746865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:45:48.746865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:48.747880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:48.749066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:45:48.749066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:45:48.749066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:45:48.749066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:45:48.749066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:45:48.749066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:45:48.749066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:45:48.749066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:45:48.749066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 632/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:45:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:48.753728 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86d626-4680-4941-ad5a-6c6ee8aa79c9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:48.852734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86d626-4680-4941-ad5a-6c6ee8aa79c9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:48.885809 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9712d1-6f61-44d1-a00e-85c08706ecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:48.886402 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9712d1-6f61-44d1-a00e-85c08706ecb2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:48.886680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9712d1-6f61-44d1-a00e-85c08706ecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:48.886953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9712d1-6f61-44d1-a00e-85c08706ecb2 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:48.887625 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9712d1-6f61-44d1-a00e-85c08706ecb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:48.887927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9712d1-6f61-44d1-a00e-85c08706ecb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:48.889384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9712d1-6f61-44d1-a00e-85c08706ecb2 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:48.901452 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 627/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:45:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:48.907452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2ebd2-d5c9-471c-96da-b39ba1f93eba None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:48.932854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:49.014801 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2ebd2-d5c9-471c-96da-b39ba1f93eba None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:49.047242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcba13f-9ad8-40bc-9621-3ef9907fd671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:49.047830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcba13f-9ad8-40bc-9621-3ef9907fd671 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:49.048121 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcba13f-9ad8-40bc-9621-3ef9907fd671 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:49.048388 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcba13f-9ad8-40bc-9621-3ef9907fd671 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:49.049092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcba13f-9ad8-40bc-9621-3ef9907fd671 None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=b16b81ab30294bc4a8c87b6d7c3573f0) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:49.049391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcba13f-9ad8-40bc-9621-3ef9907fd671 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:49.050778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcba13f-9ad8-40bc-9621-3ef9907fd671 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:49.055929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 633/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:45:48 2022] GET /identity/v3/role_assignments?user.id=b16b81ab30294bc4a8c87b6d7c3573f0&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:51.186741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bffefc-3c74-42da-82bd-d77dba33ccd7 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:51.187272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bffefc-3c74-42da-82bd-d77dba33ccd7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:51.187272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bffefc-3c74-42da-82bd-d77dba33ccd7 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:51.188650 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 628/1261] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:45:51.193948 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc05f77-7b0f-4112-9b1c-ecc465a4d716 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:51.194309 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc05f77-7b0f-4112-9b1c-ecc465a4d716 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:51.194494 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc05f77-7b0f-4112-9b1c-ecc465a4d716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:51.213482 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0bc05f77-7b0f-4112-9b1c-ecc465a4d716 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:45:51.214679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0bc05f77-7b0f-4112-9b1c-ecc465a4d716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:45:51.280504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 634/1262] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:45:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:45:51.549066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bffefc-3c74-42da-82bd-d77dba33ccd7 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:51.560052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:51.648152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bffefc-3c74-42da-82bd-d77dba33ccd7 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:51.680789 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf825e6-34dc-4cc3-85c2-4fd9258c6845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:51.681476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf825e6-34dc-4cc3-85c2-4fd9258c6845 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:51.681754 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf825e6-34dc-4cc3-85c2-4fd9258c6845 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:51.682022 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf825e6-34dc-4cc3-85c2-4fd9258c6845 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:51.691768 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf825e6-34dc-4cc3-85c2-4fd9258c6845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:45:51.698444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-cdf825e6-34dc-4cc3-85c2-4fd9258c6845 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:45:51.698444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:51.698444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:51.698444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:51.698444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:51.698444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:51.698444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:51.698444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:51.698444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:51.698444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:51.698444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:51.698444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:51.698444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:45:51.698444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:45:51.698444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:45:51.698444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:45:51.699538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:45:51.699538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:45:51.699538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:45:51.699538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:45:51.699538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:51.699538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:51.699538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:51.699538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:51.699538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:51.699538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:51.699538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:51.699538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:51.699538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:51.699538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:51.699538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:51.699538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:51.700770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:51.700770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:51.700770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:51.700770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:51.700770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:45:51.700770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:45:51.700770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:45:51.700770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:45:51.700770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:45:51.700770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:45:51.700770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:45:51.700770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:45:51.700770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 629/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:45:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:51.705441 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc05f77-7b0f-4112-9b1c-ecc465a4d716 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:51.723240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:51.802535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc05f77-7b0f-4112-9b1c-ecc465a4d716 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:51.836290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb948a16-a509-4ef8-92a6-ac77d900bb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:51.836883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb948a16-a509-4ef8-92a6-ac77d900bb13 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:51.837232 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb948a16-a509-4ef8-92a6-ac77d900bb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:51.837615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb948a16-a509-4ef8-92a6-ac77d900bb13 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:51.838445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb948a16-a509-4ef8-92a6-ac77d900bb13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:51.838751 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb948a16-a509-4ef8-92a6-ac77d900bb13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:51.840119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb948a16-a509-4ef8-92a6-ac77d900bb13 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:51.847803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 635/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:45:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:51.853846 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf825e6-34dc-4cc3-85c2-4fd9258c6845 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:51.919947 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:51.953038 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf825e6-34dc-4cc3-85c2-4fd9258c6845 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:51.992570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c7882-e653-467f-aad0-4c44391554a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:51.993310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c7882-e653-467f-aad0-4c44391554a4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:51.993519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c7882-e653-467f-aad0-4c44391554a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:51.993788 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c7882-e653-467f-aad0-4c44391554a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:52.005133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c7882-e653-467f-aad0-4c44391554a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:52.006897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c7882-e653-467f-aad0-4c44391554a4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:52.025726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-778c7882-e653-467f-aad0-4c44391554a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:52.025726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:52.025726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:52.025726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:52.025726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:52.025726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:52.025726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:52.025726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:52.025726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:52.025726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:52.025726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:52.025726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:52.025726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:45:52.025726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:45:52.025726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:45:52.025726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:52.026872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:45:52.027769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:45:52.027769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:52.027769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:45:52.027769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 630/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:52.032653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb948a16-a509-4ef8-92a6-ac77d900bb13 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:52.111717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:52.129984 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb948a16-a509-4ef8-92a6-ac77d900bb13 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:52.162314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3281877-ea69-49c0-af9c-815492749537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:52.162898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3281877-ea69-49c0-af9c-815492749537 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:52.163170 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3281877-ea69-49c0-af9c-815492749537 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:52.163435 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3281877-ea69-49c0-af9c-815492749537 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:52.164055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3281877-ea69-49c0-af9c-815492749537 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:52.164383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3281877-ea69-49c0-af9c-815492749537 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:52.165754 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3281877-ea69-49c0-af9c-815492749537 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:52.176815 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 636/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:52.182440 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c7882-e653-467f-aad0-4c44391554a4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:52.280056 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c7882-e653-467f-aad0-4c44391554a4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:52.299317 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:52.313673 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b41e92-05d5-44bf-980f-1cb86a9835f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:52.314278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b41e92-05d5-44bf-980f-1cb86a9835f2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:52.314554 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b41e92-05d5-44bf-980f-1cb86a9835f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:52.314823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b41e92-05d5-44bf-980f-1cb86a9835f2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:52.322384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b41e92-05d5-44bf-980f-1cb86a9835f2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:52.323858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b41e92-05d5-44bf-980f-1cb86a9835f2 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:52.334760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-a8b41e92-05d5-44bf-980f-1cb86a9835f2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 07:45:52.334760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:52.334760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:52.334760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:52.334760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:52.334760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:52.334760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:52.334760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:52.334760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:52.334760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:52.334760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:52.334760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:52.334760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:45:52.334760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:45:52.334760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:45:52.334760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:52.335859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:52.337093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:52.337093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:52.337093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:45:52.337093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:45:52.337093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:45:52.337093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:45:52.337093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:45:52.337093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:45:52.337093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 07:45:52.337093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:45:52.337093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 631/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:52.341681 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3281877-ea69-49c0-af9c-815492749537 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:52.438773 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3281877-ea69-49c0-af9c-815492749537 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:52.471088 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec3142-c289-47b5-a396-ce774e0562e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:52.471674 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec3142-c289-47b5-a396-ce774e0562e6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:52.471947 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec3142-c289-47b5-a396-ce774e0562e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:52.472255 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec3142-c289-47b5-a396-ce774e0562e6 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:52.473012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec3142-c289-47b5-a396-ce774e0562e6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:52.473314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec3142-c289-47b5-a396-ce774e0562e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:52.474712 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec3142-c289-47b5-a396-ce774e0562e6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:52.483353 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:52.492505 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 637/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Sep 7 07:45:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:52.499019 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b41e92-05d5-44bf-980f-1cb86a9835f2 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:52.602456 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b41e92-05d5-44bf-980f-1cb86a9835f2 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:52.641657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587335d3-528b-48cd-994d-cb65b853351b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:52.642395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587335d3-528b-48cd-994d-cb65b853351b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:52.642826 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587335d3-528b-48cd-994d-cb65b853351b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:52.643252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587335d3-528b-48cd-994d-cb65b853351b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:52.655971 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587335d3-528b-48cd-994d-cb65b853351b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:52.657931 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587335d3-528b-48cd-994d-cb65b853351b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:52.677489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-587335d3-528b-48cd-994d-cb65b853351b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:52.677489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:52.677489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:52.677489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:52.677489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:52.677489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:52.677489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:52.677489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:52.677489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:52.677489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:52.677489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:52.677489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:52.677489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:45:52.677489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:45:52.677489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:45:52.677489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:52.678617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:45:52.679672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:45:52.679672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:52.679672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:45:52.679672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 632/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:52.685482 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec3142-c289-47b5-a396-ce774e0562e6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:52.782560 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec3142-c289-47b5-a396-ce774e0562e6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:52.815556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb692165-e92e-410c-8a08-46b9561aaca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:52.816175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb692165-e92e-410c-8a08-46b9561aaca8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:52.816454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb692165-e92e-410c-8a08-46b9561aaca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:52.816737 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb692165-e92e-410c-8a08-46b9561aaca8 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:52.817351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb692165-e92e-410c-8a08-46b9561aaca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:52.817652 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb692165-e92e-410c-8a08-46b9561aaca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:52.819260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb692165-e92e-410c-8a08-46b9561aaca8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:52.830653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 638/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:52.836680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587335d3-528b-48cd-994d-cb65b853351b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:52.871617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:52.939879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587335d3-528b-48cd-994d-cb65b853351b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:52.978735 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157c5d4-d38d-494f-baae-773fbe8e0ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:52.979592 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157c5d4-d38d-494f-baae-773fbe8e0ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:52.980058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157c5d4-d38d-494f-baae-773fbe8e0ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:52.980489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157c5d4-d38d-494f-baae-773fbe8e0ca0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:52.987899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157c5d4-d38d-494f-baae-773fbe8e0ca0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:52.989409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157c5d4-d38d-494f-baae-773fbe8e0ca0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:53.001324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-9157c5d4-d38d-494f-baae-773fbe8e0ca0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:45:53.001324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:53.001324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:53.001324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:53.001324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:53.001324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:53.001324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:53.001324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:53.001324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:53.001324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:53.001324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:53.001324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:53.001324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:45:53.001324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:45:53.001324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:53.002269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:53.003207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:45:53.003207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:45:53.003207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:45:53.003207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:45:53.003207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:45:53.003207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:45:53.003207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:45:53.003207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:45:53.003207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 633/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:45:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:53.008327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb692165-e92e-410c-8a08-46b9561aaca8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:53.066316 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:53.108432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb692165-e92e-410c-8a08-46b9561aaca8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:53.141250 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236df09-0020-40b1-a64e-047db6c1c65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:53.141843 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236df09-0020-40b1-a64e-047db6c1c65b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:53.142122 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236df09-0020-40b1-a64e-047db6c1c65b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:53.142395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236df09-0020-40b1-a64e-047db6c1c65b None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:53.143075 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e236df09-0020-40b1-a64e-047db6c1c65b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:53.143381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e236df09-0020-40b1-a64e-047db6c1c65b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:53.144837 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e236df09-0020-40b1-a64e-047db6c1c65b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:53.156713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 639/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:45:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:53.162599 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157c5d4-d38d-494f-baae-773fbe8e0ca0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:53.254835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:53.265879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157c5d4-d38d-494f-baae-773fbe8e0ca0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:53.303867 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b3986-afb8-439f-a56e-3a0009c694f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:53.304557 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b3986-afb8-439f-a56e-3a0009c694f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:53.304833 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b3986-afb8-439f-a56e-3a0009c694f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:53.305101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b3986-afb8-439f-a56e-3a0009c694f9 None admin] PATH_INFO: `/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/b16b81ab30294bc4a8c87b6d7c3573f0/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:53.308108 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b3986-afb8-439f-a56e-3a0009c694f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a949b959e584474aafb4a05ebbabcc4, user_id=b16b81ab30294bc4a8c87b6d7c3573f0, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:53.309552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b3986-afb8-439f-a56e-3a0009c694f9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:53.323122 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 634/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 7 07:45:53 2022] PUT /identity/v3/projects/9a949b959e584474aafb4a05ebbabcc4/users/b16b81ab30294bc4a8c87b6d7c3573f0/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:45:55.431394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673e4ae-ecc9-402f-99c6-7a3b433d9f66 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:55.431896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673e4ae-ecc9-402f-99c6-7a3b433d9f66 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:55.431896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673e4ae-ecc9-402f-99c6-7a3b433d9f66 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:55.433282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 640/1274] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:45:55.438464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaa422-71c7-46e8-ba36-f402f55ef8a2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:55.438882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaa422-71c7-46e8-ba36-f402f55ef8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:55.439000 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaa422-71c7-46e8-ba36-f402f55ef8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:55.457229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1ebaa422-71c7-46e8-ba36-f402f55ef8a2 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:45:55.458392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1ebaa422-71c7-46e8-ba36-f402f55ef8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:45:55.522535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 635/1275] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:45:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:45:55.794561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673e4ae-ecc9-402f-99c6-7a3b433d9f66 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:55.900224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673e4ae-ecc9-402f-99c6-7a3b433d9f66 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:55.917427 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:55.934281 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dec756-0008-474f-9b86-38e21703c65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:55.934907 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dec756-0008-474f-9b86-38e21703c65f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:55.935186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dec756-0008-474f-9b86-38e21703c65f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:55.935460 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dec756-0008-474f-9b86-38e21703c65f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:55.945509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dec756-0008-474f-9b86-38e21703c65f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 07:45:55.951687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-60dec756-0008-474f-9b86-38e21703c65f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:45:55.951687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:55.951687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:55.951687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:55.951687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:55.951687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:55.951687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:55.951687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:55.951687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:55.951687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:55.951687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:55.951687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:55.951687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 07:45:55.951687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 07:45:55.951687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 07:45:55.951687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 07:45:55.953206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 07:45:55.953206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 07:45:55.953206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 07:45:55.953206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 07:45:55.953206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:55.953206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:55.953206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:55.953206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:55.953206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:55.953206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:55.953206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:55.953206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:55.953206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:55.953206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:55.953206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:55.953206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:55.954502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:55.954502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:55.954502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:55.954502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:55.954502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 07:45:55.954502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 07:45:55.954502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 07:45:55.954502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 07:45:55.954502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 07:45:55.954502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 07:45:55.954502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 07:45:55.954502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:45:55.954502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 641/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 7 07:45:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:55.960124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebaa422-71c7-46e8-ba36-f402f55ef8a2 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:56.050433 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:56.068684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebaa422-71c7-46e8-ba36-f402f55ef8a2 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:56.102533 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393a5f3-93cb-46ea-bffe-fa96608c8bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:56.103526 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393a5f3-93cb-46ea-bffe-fa96608c8bfe None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:56.103810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393a5f3-93cb-46ea-bffe-fa96608c8bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:56.104095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393a5f3-93cb-46ea-bffe-fa96608c8bfe None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:56.104755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393a5f3-93cb-46ea-bffe-fa96608c8bfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:56.105051 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393a5f3-93cb-46ea-bffe-fa96608c8bfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:56.106414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393a5f3-93cb-46ea-bffe-fa96608c8bfe None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:56.114241 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 636/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 7 07:45:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:56.120170 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dec756-0008-474f-9b86-38e21703c65f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:56.217986 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dec756-0008-474f-9b86-38e21703c65f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:56.250772 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64d4eb-8e8e-43a9-bbe6-9f53c7ba2009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:56.251385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64d4eb-8e8e-43a9-bbe6-9f53c7ba2009 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:56.251663 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64d4eb-8e8e-43a9-bbe6-9f53c7ba2009 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:56.251933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64d4eb-8e8e-43a9-bbe6-9f53c7ba2009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:56.262177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64d4eb-8e8e-43a9-bbe6-9f53c7ba2009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:56.263603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64d4eb-8e8e-43a9-bbe6-9f53c7ba2009 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:56.277746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-7a64d4eb-8e8e-43a9-bbe6-9f53c7ba2009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:56.277746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:56.277746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:56.277746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:56.277746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:56.277746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:56.277746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:56.277746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:56.277746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:56.277746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:56.277746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:56.277746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:56.277746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:45:56.277746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:45:56.277746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:45:56.277746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:56.278932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:45:56.279870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:45:56.279870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:56.279870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:45:56.279870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 642/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:56.284642 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393a5f3-93cb-46ea-bffe-fa96608c8bfe None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:56.386444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393a5f3-93cb-46ea-bffe-fa96608c8bfe None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:56.419528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec0dfa-23ff-4cba-953e-4904c5827edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:56.420159 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec0dfa-23ff-4cba-953e-4904c5827edf None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:56.420431 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec0dfa-23ff-4cba-953e-4904c5827edf None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:56.420698 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec0dfa-23ff-4cba-953e-4904c5827edf None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:56.421324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec0dfa-23ff-4cba-953e-4904c5827edf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:56.421645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec0dfa-23ff-4cba-953e-4904c5827edf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:56.423014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec0dfa-23ff-4cba-953e-4904c5827edf None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:56.435217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 637/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:56.440978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64d4eb-8e8e-43a9-bbe6-9f53c7ba2009 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:56.472527 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:56.540267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64d4eb-8e8e-43a9-bbe6-9f53c7ba2009 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:56.573092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bc790-f894-491f-9eb2-267c6b8558b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:56.573696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bc790-f894-491f-9eb2-267c6b8558b4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:56.573965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bc790-f894-491f-9eb2-267c6b8558b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:56.574229 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bc790-f894-491f-9eb2-267c6b8558b4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:56.581249 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bc790-f894-491f-9eb2-267c6b8558b4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:56.582649 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bc790-f894-491f-9eb2-267c6b8558b4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:56.593478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-174bc790-f894-491f-9eb2-267c6b8558b4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 07:45:56.593478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:56.593478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:56.593478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:56.593478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:56.593478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:56.593478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:56.593478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:56.593478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:56.593478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:56.593478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:56.593478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:56.593478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 07:45:56.593478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 07:45:56.593478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 07:45:56.593478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:56.594567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:56.595496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:56.595496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:56.595496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 07:45:56.595496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 07:45:56.595496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 07:45:56.595496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 07:45:56.595496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 07:45:56.595496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 07:45:56.595496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 07:45:56.595496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:45:56.595496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 643/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:56.600441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec0dfa-23ff-4cba-953e-4904c5827edf None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:56.604441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:56.706740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec0dfa-23ff-4cba-953e-4904c5827edf None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:56.739344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257b806-9a81-4e1e-9b07-542035e3e785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:56.739934 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257b806-9a81-4e1e-9b07-542035e3e785 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:56.740240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257b806-9a81-4e1e-9b07-542035e3e785 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:56.740511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257b806-9a81-4e1e-9b07-542035e3e785 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:56.741220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257b806-9a81-4e1e-9b07-542035e3e785 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:56.741521 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257b806-9a81-4e1e-9b07-542035e3e785 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:56.742945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257b806-9a81-4e1e-9b07-542035e3e785 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:56.760274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 638/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Sep 7 07:45:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:56.766154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bc790-f894-491f-9eb2-267c6b8558b4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:56.852930 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:56.862651 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bc790-f894-491f-9eb2-267c6b8558b4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:56.894471 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ff51f-d6a4-4237-9ba8-553f6029b47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:56.895050 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ff51f-d6a4-4237-9ba8-553f6029b47d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:56.895323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ff51f-d6a4-4237-9ba8-553f6029b47d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:56.895587 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ff51f-d6a4-4237-9ba8-553f6029b47d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:56.959944 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ff51f-d6a4-4237-9ba8-553f6029b47d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:56.961341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ff51f-d6a4-4237-9ba8-553f6029b47d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:57.044894 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:57.054738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 644/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 7 07:45:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 07:45:57.061039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257b806-9a81-4e1e-9b07-542035e3e785 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:57.120430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:57.158065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257b806-9a81-4e1e-9b07-542035e3e785 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:57.191296 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280aa552-347d-4fbb-85c1-b93f1af414a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:57.191913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280aa552-347d-4fbb-85c1-b93f1af414a7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:57.192220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280aa552-347d-4fbb-85c1-b93f1af414a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:57.192490 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280aa552-347d-4fbb-85c1-b93f1af414a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:57.203025 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280aa552-347d-4fbb-85c1-b93f1af414a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:57.204484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280aa552-347d-4fbb-85c1-b93f1af414a7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:57.218631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-280aa552-347d-4fbb-85c1-b93f1af414a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:57.218631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:57.218631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:57.218631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:57.218631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:57.218631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:57.218631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:57.218631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:57.218631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:57.218631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:57.218631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:57.218631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:57.218631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 07:45:57.218631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 07:45:57.218631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 07:45:57.218631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:57.219732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 07:45:57.220979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 07:45:57.220979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 07:45:57.220979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:45:57.220979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 639/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:57.225212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ff51f-d6a4-4237-9ba8-553f6029b47d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:57.319529 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ff51f-d6a4-4237-9ba8-553f6029b47d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:57.352074 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09bec1-a02b-4eba-82de-10f416f9da32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:57.352819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09bec1-a02b-4eba-82de-10f416f9da32 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:57.353100 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09bec1-a02b-4eba-82de-10f416f9da32 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:57.353361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09bec1-a02b-4eba-82de-10f416f9da32 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:57.353976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09bec1-a02b-4eba-82de-10f416f9da32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:57.354270 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09bec1-a02b-4eba-82de-10f416f9da32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:57.355634 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09bec1-a02b-4eba-82de-10f416f9da32 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:57.366214 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 645/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 7 07:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:57.372279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280aa552-347d-4fbb-85c1-b93f1af414a7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:57.470583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280aa552-347d-4fbb-85c1-b93f1af414a7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:57.484061 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:57.504318 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840198db-efbb-4bf4-b48f-bb6080fc3c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:57.504932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840198db-efbb-4bf4-b48f-bb6080fc3c42 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:57.505205 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840198db-efbb-4bf4-b48f-bb6080fc3c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:57.505471 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840198db-efbb-4bf4-b48f-bb6080fc3c42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:57.511986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840198db-efbb-4bf4-b48f-bb6080fc3c42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:57.513503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840198db-efbb-4bf4-b48f-bb6080fc3c42 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:57.524509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-840198db-efbb-4bf4-b48f-bb6080fc3c42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:45:57.524509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:45:57.524509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:45:57.524509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:45:57.524509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:45:57.524509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:45:57.524509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:45:57.524509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:45:57.524509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:45:57.524509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:45:57.524509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:45:57.524509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:45:57.524509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 07:45:57.524509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 07:45:57.524509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:45:57.530164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:45:57.531220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 07:45:57.531220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 07:45:57.531220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 07:45:57.531220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 07:45:57.531220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 07:45:57.531220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 07:45:57.531220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 07:45:57.531220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Sep 07 07:45:57.531220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 640/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 7 07:45:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:45:57.531701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09bec1-a02b-4eba-82de-10f416f9da32 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:57.580126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:45:57.628301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09bec1-a02b-4eba-82de-10f416f9da32 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:57.659776 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078c4f0-4efc-4eb9-99d4-b3066225b727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:57.660393 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078c4f0-4efc-4eb9-99d4-b3066225b727 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:57.660672 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078c4f0-4efc-4eb9-99d4-b3066225b727 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:57.660942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078c4f0-4efc-4eb9-99d4-b3066225b727 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:57.661613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078c4f0-4efc-4eb9-99d4-b3066225b727 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:57.661914 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078c4f0-4efc-4eb9-99d4-b3066225b727 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:57.663306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078c4f0-4efc-4eb9-99d4-b3066225b727 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:57.675757 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 646/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 7 07:45:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:45:57.681763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840198db-efbb-4bf4-b48f-bb6080fc3c42 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:45:57.781774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840198db-efbb-4bf4-b48f-bb6080fc3c42 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:45:57.813700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb3ece-7e7c-42ca-8411-6505cb842e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:45:57.814288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb3ece-7e7c-42ca-8411-6505cb842e5a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:45:57.814555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb3ece-7e7c-42ca-8411-6505cb842e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:45:57.814815 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb3ece-7e7c-42ca-8411-6505cb842e5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:45:57.815509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb3ece-7e7c-42ca-8411-6505cb842e5a None admin] RBAC: Adding query filter params (role.id=202f2746fd624f55ad25e548a4db9ed8, scope.project.id=9a949b959e584474aafb4a05ebbabcc4, user.id=b16b81ab30294bc4a8c87b6d7c3573f0) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:45:57.815808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb3ece-7e7c-42ca-8411-6505cb842e5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:45:57.817238 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb3ece-7e7c-42ca-8411-6505cb842e5a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:45:57.823088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 641/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 7 07:45:57 2022] GET /identity/v3/role_assignments?user.id=b16b81ab30294bc4a8c87b6d7c3573f0&scope.project.id=9a949b959e584474aafb4a05ebbabcc4&role.id=202f2746fd624f55ad25e548a4db9ed8 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:46:00.135512 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db0069-36e8-4f78-9815-aa123119cd9b None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:00.136070 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db0069-36e8-4f78-9815-aa123119cd9b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:00.136070 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db0069-36e8-4f78-9815-aa123119cd9b None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:00.137394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 647/1288] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:46:00.142437 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5e669-d174-493a-aaac-f0bb12fa2d72 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:00.142803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5e669-d174-493a-aaac-f0bb12fa2d72 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:00.142981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5e669-d174-493a-aaac-f0bb12fa2d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:00.161282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-02b5e669-d174-493a-aaac-f0bb12fa2d72 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:46:00.162464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-02b5e669-d174-493a-aaac-f0bb12fa2d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:46:00.191561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:46:00.229237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 642/1289] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:46:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:46:00.495367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db0069-36e8-4f78-9815-aa123119cd9b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:46:00.602288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db0069-36e8-4f78-9815-aa123119cd9b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:46:00.612703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:46:00.635630 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737dd29-53e0-4ac7-95e0-072b4bfae115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:46:00.636278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737dd29-53e0-4ac7-95e0-072b4bfae115 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:00.636552 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737dd29-53e0-4ac7-95e0-072b4bfae115 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:00.636819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737dd29-53e0-4ac7-95e0-072b4bfae115 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:00.637384 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737dd29-53e0-4ac7-95e0-072b4bfae115 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:46:00.638717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737dd29-53e0-4ac7-95e0-072b4bfae115 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-3737dd29-53e0-4ac7-95e0-072b4bfae115 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:46:00.647092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:46:00.648333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:46:00.648333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:46:00.648333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:46:00.648333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:46:00.648333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:46:00.648333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:46:00.648333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:46:00.648333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:46:00.648333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:46:00.648333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:46:00.648333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:46:00.648333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:46:00.648333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:46:00.648333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:46:00.648333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:46:00.648333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:46:00.650179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:46:00.650179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:46:00.650179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:46:00.650179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:46:00.650179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:46:00.650179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:46:00.650179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:46:00.650179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:46:00.650179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:46:00.650179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:46:00.650179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:46:00.650179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:46:00.650179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 07:46:00.650179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:46:00.650179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 648/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 7 07:46:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:46:00.654617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5e669-d174-493a-aaac-f0bb12fa2d72 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:46:00.749281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5e669-d174-493a-aaac-f0bb12fa2d72 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:46:00.781760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d4878-81a4-4b59-bad2-c8d491647103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:46:00.782343 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d4878-81a4-4b59-bad2-c8d491647103 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:00.782612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d4878-81a4-4b59-bad2-c8d491647103 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:00.782875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d4878-81a4-4b59-bad2-c8d491647103 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:00.783479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d4878-81a4-4b59-bad2-c8d491647103 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:46:00.783774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d4878-81a4-4b59-bad2-c8d491647103 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:46:00.785145 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d4878-81a4-4b59-bad2-c8d491647103 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:46:00.789762 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 643/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Sep 7 07:46:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:46:00.795167 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737dd29-53e0-4ac7-95e0-072b4bfae115 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:46:00.896353 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737dd29-53e0-4ac7-95e0-072b4bfae115 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:46:00.928156 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe7523-c34d-4ee2-ab75-ee59021b6f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:46:00.928745 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe7523-c34d-4ee2-ab75-ee59021b6f52 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:00.929024 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe7523-c34d-4ee2-ab75-ee59021b6f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:00.929296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe7523-c34d-4ee2-ab75-ee59021b6f52 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:00.929911 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe7523-c34d-4ee2-ab75-ee59021b6f52 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:46:00.930212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe7523-c34d-4ee2-ab75-ee59021b6f52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:46:00.932112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe7523-c34d-4ee2-ab75-ee59021b6f52 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:46:00.936371 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 649/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Sep 7 07:46:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:46:03.075173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247c70c-8f94-4dd1-978b-8dedb88d9923 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:03.075700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247c70c-8f94-4dd1-978b-8dedb88d9923 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:03.075700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247c70c-8f94-4dd1-978b-8dedb88d9923 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:03.077064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 644/1293] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:46:03.082293 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03652542-e582-441a-b445-f75efd2e066e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:03.082690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03652542-e582-441a-b445-f75efd2e066e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:03.082842 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03652542-e582-441a-b445-f75efd2e066e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:03.101413 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-03652542-e582-441a-b445-f75efd2e066e None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:46:03.103209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-03652542-e582-441a-b445-f75efd2e066e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:46:03.131346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:46:03.168261 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 650/1294] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:46:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:46:03.432944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247c70c-8f94-4dd1-978b-8dedb88d9923 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:46:03.463325 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:46:03.531519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247c70c-8f94-4dd1-978b-8dedb88d9923 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:46:03.564714 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba23c2c-8f08-4054-b5ab-0d5e6270972d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:46:03.565301 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba23c2c-8f08-4054-b5ab-0d5e6270972d None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:03.565573 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba23c2c-8f08-4054-b5ab-0d5e6270972d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:03.565839 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba23c2c-8f08-4054-b5ab-0d5e6270972d None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:03.566464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba23c2c-8f08-4054-b5ab-0d5e6270972d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:46:03.568098 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba23c2c-8f08-4054-b5ab-0d5e6270972d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:46:03.582450 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 645/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 7 07:46:03 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:46:05.748427 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bab76-aee0-40da-b388-d5f932e0fb95 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:05.748944 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bab76-aee0-40da-b388-d5f932e0fb95 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:05.748944 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bab76-aee0-40da-b388-d5f932e0fb95 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:05.750225 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 651/1296] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:46:05.755316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b9c5a-7799-4cf1-b107-c8319edb1f4b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:05.755697 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b9c5a-7799-4cf1-b107-c8319edb1f4b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:05.755892 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b9c5a-7799-4cf1-b107-c8319edb1f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:05.774902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-8a7b9c5a-7799-4cf1-b107-c8319edb1f4b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:46:05.776069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8a7b9c5a-7799-4cf1-b107-c8319edb1f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:46:05.846879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 646/1297] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:46:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:46:06.110405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bab76-aee0-40da-b388-d5f932e0fb95 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:46:06.216736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bab76-aee0-40da-b388-d5f932e0fb95 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:46:06.251347 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c00a5a-c691-465a-ba57-22b681ade625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:46:06.251998 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c00a5a-c691-465a-ba57-22b681ade625 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:06.252342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c00a5a-c691-465a-ba57-22b681ade625 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:06.252653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c00a5a-c691-465a-ba57-22b681ade625 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:06.253265 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c00a5a-c691-465a-ba57-22b681ade625 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:46:06.254750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c00a5a-c691-465a-ba57-22b681ade625 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-40c00a5a-c691-465a-ba57-22b681ade625 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:46:06.265154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:46:06.266409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:46:06.266409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:46:06.266409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:46:06.266409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:46:06.266409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:46:06.266409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:46:06.266409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:46:06.266409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:46:06.266409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:46:06.266409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:46:06.266409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:46:06.266409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:46:06.266409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:46:06.266409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:46:06.266409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:46:06.266409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:46:06.267456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:46:06.267456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:46:06.267456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:46:06.267456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:46:06.267456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:46:06.267456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:46:06.267456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:46:06.267456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:46:06.267456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:46:06.267456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:46:06.267456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:46:06.267456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:46:06.267456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 07:46:06.267456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:46:06.267456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 652/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 7 07:46:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:46:06.274772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b9c5a-7799-4cf1-b107-c8319edb1f4b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:46:06.293980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:46:06.378625 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b9c5a-7799-4cf1-b107-c8319edb1f4b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:46:06.415795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1654b5-287a-4533-a67a-a7d922508037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:46:06.416476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1654b5-287a-4533-a67a-a7d922508037 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:06.416781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1654b5-287a-4533-a67a-a7d922508037 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:06.417056 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1654b5-287a-4533-a67a-a7d922508037 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:06.417700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1654b5-287a-4533-a67a-a7d922508037 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:46:06.418042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1654b5-287a-4533-a67a-a7d922508037 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:46:06.419503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1654b5-287a-4533-a67a-a7d922508037 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:46:06.424933 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 647/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Sep 7 07:46:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:46:06.431543 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c00a5a-c691-465a-ba57-22b681ade625 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:46:06.462296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:46:06.527943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c00a5a-c691-465a-ba57-22b681ade625 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:46:06.560512 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4bc481-55cd-41f2-8a7b-1b432d3eee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:46:06.561094 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4bc481-55cd-41f2-8a7b-1b432d3eee1f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:06.561367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4bc481-55cd-41f2-8a7b-1b432d3eee1f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:06.561631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4bc481-55cd-41f2-8a7b-1b432d3eee1f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:06.562316 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4bc481-55cd-41f2-8a7b-1b432d3eee1f None admin] RBAC: Adding query filter params (interface=public, service_id=e96e73311fa54df4ab3f25170d737937, region_id=RegionOne) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:46:06.562620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4bc481-55cd-41f2-8a7b-1b432d3eee1f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:46:06.563960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4bc481-55cd-41f2-8a7b-1b432d3eee1f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:46:06.569003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 653/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Sep 7 07:46:06 2022] GET /identity/v3/endpoints?service_id=e96e73311fa54df4ab3f25170d737937&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:46:06.575244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1654b5-287a-4533-a67a-a7d922508037 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:46:06.660044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:46:06.679704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1654b5-287a-4533-a67a-a7d922508037 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:46:06.716194 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b99535-28a8-488f-b97b-cfea69326d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:46:06.716818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b99535-28a8-488f-b97b-cfea69326d09 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:06.717121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b99535-28a8-488f-b97b-cfea69326d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:06.717394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b99535-28a8-488f-b97b-cfea69326d09 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:06.718041 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b99535-28a8-488f-b97b-cfea69326d09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:46:06.719461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b99535-28a8-488f-b97b-cfea69326d09 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:46:06.726308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 648/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 7 07:46:06 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:46:08.832223 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5581b-ccf0-4401-a66b-425fdb0279a0 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:08.832844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5581b-ccf0-4401-a66b-425fdb0279a0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:08.832844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5581b-ccf0-4401-a66b-425fdb0279a0 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:08.833994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 654/1302] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:46:08.839107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27ce32-ad04-4b1f-8db1-b2f4d1c5b778 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:08.839511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27ce32-ad04-4b1f-8db1-b2f4d1c5b778 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:08.839654 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27ce32-ad04-4b1f-8db1-b2f4d1c5b778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:08.858030 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-fd27ce32-ad04-4b1f-8db1-b2f4d1c5b778 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:46:08.859201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fd27ce32-ad04-4b1f-8db1-b2f4d1c5b778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:46:08.923664 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 649/1303] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:46:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 07:46:09.193125 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5581b-ccf0-4401-a66b-425fdb0279a0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:46:09.285870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:46:09.298168 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5581b-ccf0-4401-a66b-425fdb0279a0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:46:09.330321 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb818ce-5c2f-419a-97cc-dcbdeafd866b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:46:09.330934 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb818ce-5c2f-419a-97cc-dcbdeafd866b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:09.331205 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb818ce-5c2f-419a-97cc-dcbdeafd866b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:09.331469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb818ce-5c2f-419a-97cc-dcbdeafd866b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:09.332035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb818ce-5c2f-419a-97cc-dcbdeafd866b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:46:09.333386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb818ce-5c2f-419a-97cc-dcbdeafd866b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-fcb818ce-5c2f-419a-97cc-dcbdeafd866b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 07:46:09.342267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 07:46:09.343487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 07:46:09.343487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 07:46:09.343487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 07:46:09.343487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 07:46:09.343487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 07:46:09.343487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 07:46:09.343487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 07:46:09.343487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 07:46:09.343487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 07:46:09.343487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 07:46:09.343487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 07:46:09.343487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 07:46:09.343487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 07:46:09.343487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 07:46:09.343487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 07:46:09.343487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 07:46:09.344504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 07:46:09.344504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Sep 07 07:46:09.344504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 07:46:09.344504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Sep 07 07:46:09.344504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 07:46:09.344504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 07:46:09.344504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 07:46:09.344504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Sep 07 07:46:09.344504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 07:46:09.344504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Sep 07 07:46:09.344504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 07:46:09.344504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 07:46:09.344504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 07:46:09.344504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Sep 07 07:46:09.344504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 655/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 7 07:46:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 07:46:09.349280 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27ce32-ad04-4b1f-8db1-b2f4d1c5b778 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:46:09.424431 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:46:09.447847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27ce32-ad04-4b1f-8db1-b2f4d1c5b778 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:46:09.480844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd39b9-f056-4764-95da-af7436fda12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:46:09.481448 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd39b9-f056-4764-95da-af7436fda12e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:09.481720 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd39b9-f056-4764-95da-af7436fda12e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:09.481981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd39b9-f056-4764-95da-af7436fda12e None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:09.482590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd39b9-f056-4764-95da-af7436fda12e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:46:09.482883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd39b9-f056-4764-95da-af7436fda12e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:46:09.484287 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd39b9-f056-4764-95da-af7436fda12e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:46:09.489231 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 650/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Sep 7 07:46:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:46:09.495029 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb818ce-5c2f-419a-97cc-dcbdeafd866b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:46:09.592036 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb818ce-5c2f-419a-97cc-dcbdeafd866b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:46:09.623951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f619b-ef01-4623-88b3-2ee2abc31927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:46:09.624688 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f619b-ef01-4623-88b3-2ee2abc31927 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:46:09.624961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f619b-ef01-4623-88b3-2ee2abc31927 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:46:09.625226 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f619b-ef01-4623-88b3-2ee2abc31927 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:46:09.625879 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f619b-ef01-4623-88b3-2ee2abc31927 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:46:09.627216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f619b-ef01-4623-88b3-2ee2abc31927 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:46:09.646484 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 656/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:46:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:46:19.486705 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=30 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:46:19.642704 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:14.046817 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d61725-1d31-4826-a66c-62fd5a15a7a8 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:14.047395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d61725-1d31-4826-a66c-62fd5a15a7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:14.047395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d61725-1d31-4826-a66c-62fd5a15a7a8 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:14.052881 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 651/1307] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:47:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:47:14.054504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae710c1-cd50-4cb2-b071-a7bd38347fbe None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:14.054956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae710c1-cd50-4cb2-b071-a7bd38347fbe None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:14.055059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae710c1-cd50-4cb2-b071-a7bd38347fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:14.149141 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-dae710c1-cd50-4cb2-b071-a7bd38347fbe None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:47:14.150365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-dae710c1-cd50-4cb2-b071-a7bd38347fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:14.276689 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 657/1308] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:47:14.580372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbaa16-983d-4eb6-b66b-421e06e83732 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:14.581011 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbaa16-983d-4eb6-b66b-421e06e83732 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:14.581498 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbaa16-983d-4eb6-b66b-421e06e83732 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:14.583559 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 652/1309] 10.222.0.26 () {54 vars in 965 bytes} [Wed Sep 7 07:47:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:47:14.590278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd4d3f-44db-437d-9ded-dd33571bcc37 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:14.590582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd4d3f-44db-437d-9ded-dd33571bcc37 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:14.590880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd4d3f-44db-437d-9ded-dd33571bcc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:14.687193 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-adfd4d3f-44db-437d-9ded-dd33571bcc37 None None] MFA Rules not processed for user `330a38822d33439191773e9a8b4cbe04`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:47:14.688521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-adfd4d3f-44db-437d-9ded-dd33571bcc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:14.712796 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:14.790447 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbaa16-983d-4eb6-b66b-421e06e83732 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:47:14.792413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0bfbaa16-983d-4eb6-b66b-421e06e83732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:14.797247 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 658/1310] 10.222.0.26 () {58 vars in 1062 bytes} [Wed Sep 7 07:47:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:47:14.928704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbaa16-983d-4eb6-b66b-421e06e83732 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:47:15.021603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb8dbd-50af-455a-945e-352a3086179b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:47:15.022213 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb8dbd-50af-455a-945e-352a3086179b service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:15.022493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb8dbd-50af-455a-945e-352a3086179b service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:15.022764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb8dbd-50af-455a-945e-352a3086179b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:15.024407 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-39eb8dbd-50af-455a-945e-352a3086179b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:15.086585 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:15.150690 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb8dbd-50af-455a-945e-352a3086179b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:47:15.152280 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb8dbd-50af-455a-945e-352a3086179b service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:47:15.175455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 653/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:47:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 07:47:17.741388 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc959d-fce9-40f7-898e-2b24283f013c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:17.741911 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc959d-fce9-40f7-898e-2b24283f013c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:17.741911 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc959d-fce9-40f7-898e-2b24283f013c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:17.743177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 659/1312] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:47:17.748614 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc69b52-97c0-4efc-b1c8-ea4d62de3e12 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:17.749047 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc69b52-97c0-4efc-b1c8-ea4d62de3e12 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:17.749166 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc69b52-97c0-4efc-b1c8-ea4d62de3e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:17.819619 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-5dc69b52-97c0-4efc-b1c8-ea4d62de3e12 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:47:17.820850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5dc69b52-97c0-4efc-b1c8-ea4d62de3e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:17.916067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 654/1313] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:47:18.210496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00065d12-f21d-40b2-b9db-245a95ec16ef None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:18.210780 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00065d12-f21d-40b2-b9db-245a95ec16ef None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:18.211053 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00065d12-f21d-40b2-b9db-245a95ec16ef None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:18.212421 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 660/1314] 10.222.0.26 () {54 vars in 965 bytes} [Wed Sep 7 07:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:47:18.218311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb1194-723c-4b78-86f1-f9d6ca9d3e97 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:18.218595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb1194-723c-4b78-86f1-f9d6ca9d3e97 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:18.218868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb1194-723c-4b78-86f1-f9d6ca9d3e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:18.230527 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:18.295105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-dbfb1194-723c-4b78-86f1-f9d6ca9d3e97 None None] MFA Rules not processed for user `330a38822d33439191773e9a8b4cbe04`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:47:18.296335 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dbfb1194-723c-4b78-86f1-f9d6ca9d3e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:18.371397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 655/1315] 10.222.0.26 () {58 vars in 1062 bytes} [Wed Sep 7 07:47:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:47:18.381307 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00065d12-f21d-40b2-b9db-245a95ec16ef None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:47:18.383547 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-00065d12-f21d-40b2-b9db-245a95ec16ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:18.485261 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00065d12-f21d-40b2-b9db-245a95ec16ef None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:47:18.553918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:18.556846 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d507b-5e3a-47eb-bba5-31f66cdcd94c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:47:18.557481 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d507b-5e3a-47eb-bba5-31f66cdcd94c service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:18.557927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d507b-5e3a-47eb-bba5-31f66cdcd94c service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:18.558378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d507b-5e3a-47eb-bba5-31f66cdcd94c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:18.560024 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4e5d507b-5e3a-47eb-bba5-31f66cdcd94c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:18.665838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d507b-5e3a-47eb-bba5-31f66cdcd94c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:47:18.667237 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d507b-5e3a-47eb-bba5-31f66cdcd94c service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:47:18.685050 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 661/1316] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:47:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:47:21.393866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7642a3-e3dc-4df7-abd4-d8b5e2dc6daa None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:21.394308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7642a3-e3dc-4df7-abd4-d8b5e2dc6daa None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:21.394402 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7642a3-e3dc-4df7-abd4-d8b5e2dc6daa None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:21.396070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 656/1317] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:47:21.401176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afe282-a603-4b3f-a9bc-0fefc935ec69 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:21.401603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afe282-a603-4b3f-a9bc-0fefc935ec69 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:21.401708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afe282-a603-4b3f-a9bc-0fefc935ec69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:21.467406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-87afe282-a603-4b3f-a9bc-0fefc935ec69 None None] MFA Rules not processed for user `3d9981f1b2c1421b8d5a89485cc87def`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:47:21.468623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-87afe282-a603-4b3f-a9bc-0fefc935ec69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:21.518395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:21.562607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 662/1318] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:47:21 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:47:21.851650 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7642a3-e3dc-4df7-abd4-d8b5e2dc6daa None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:47:21.853260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9e7642a3-e3dc-4df7-abd4-d8b5e2dc6daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:21.956831 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7642a3-e3dc-4df7-abd4-d8b5e2dc6daa None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:47:21.959978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:22.029937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be43b5-10c6-4051-9580-d4e0fbb4b46f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:47:22.030533 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be43b5-10c6-4051-9580-d4e0fbb4b46f service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:22.030809 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be43b5-10c6-4051-9580-d4e0fbb4b46f service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:22.031078 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be43b5-10c6-4051-9580-d4e0fbb4b46f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:22.032688 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a7be43b5-10c6-4051-9580-d4e0fbb4b46f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:22.140476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be43b5-10c6-4051-9580-d4e0fbb4b46f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:47:22.141939 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be43b5-10c6-4051-9580-d4e0fbb4b46f service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:47:22.159510 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 657/1319] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:47:21 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:47:24.835494 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087788e-3765-487b-af99-e6fcd44261b9 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:24.836014 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087788e-3765-487b-af99-e6fcd44261b9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:24.836014 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087788e-3765-487b-af99-e6fcd44261b9 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:24.837345 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 663/1320] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:47:24.842452 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693f7e0-48d4-44d6-a1fc-622fdd4b9d84 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:24.842858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693f7e0-48d4-44d6-a1fc-622fdd4b9d84 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:24.843002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693f7e0-48d4-44d6-a1fc-622fdd4b9d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:24.884017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:24.909844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c693f7e0-48d4-44d6-a1fc-622fdd4b9d84 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:47:24.911784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c693f7e0-48d4-44d6-a1fc-622fdd4b9d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:25.016489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 658/1321] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:47:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:47:25.302119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087788e-3765-487b-af99-e6fcd44261b9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:47:25.303681 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5087788e-3765-487b-af99-e6fcd44261b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:25.404353 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087788e-3765-487b-af99-e6fcd44261b9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:47:25.476194 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be58278-d38e-4943-9a93-840efc9f817e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:47:25.476789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be58278-d38e-4943-9a93-840efc9f817e service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:25.477065 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be58278-d38e-4943-9a93-840efc9f817e service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:25.477332 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be58278-d38e-4943-9a93-840efc9f817e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:25.478906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9be58278-d38e-4943-9a93-840efc9f817e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:25.486081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:25.584309 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be58278-d38e-4943-9a93-840efc9f817e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:47:25.585651 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be58278-d38e-4943-9a93-840efc9f817e service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:47:25.603632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 664/1322] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:47:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:47:28.403664 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d5b21-9220-4d50-8a95-516a5f9fa724 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:28.404172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d5b21-9220-4d50-8a95-516a5f9fa724 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:28.404297 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d5b21-9220-4d50-8a95-516a5f9fa724 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:28.405586 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 659/1323] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:47:28.411372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5bff4-7792-4789-9057-a45132cec81d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:28.411792 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5bff4-7792-4789-9057-a45132cec81d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:28.411921 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5bff4-7792-4789-9057-a45132cec81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:28.481869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-31d5bff4-7792-4789-9057-a45132cec81d None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:47:28.483130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-31d5bff4-7792-4789-9057-a45132cec81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:28.545425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:28.590437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 665/1324] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:47:28.874773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d5b21-9220-4d50-8a95-516a5f9fa724 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:47:28.876327 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-068d5b21-9220-4d50-8a95-516a5f9fa724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:28.985338 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d5b21-9220-4d50-8a95-516a5f9fa724 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:47:28.998374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:29.060816 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bef96a-01f7-4559-a4c6-3f50e06e5126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:47:29.061420 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bef96a-01f7-4559-a4c6-3f50e06e5126 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:29.061700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bef96a-01f7-4559-a4c6-3f50e06e5126 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:29.061971 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bef96a-01f7-4559-a4c6-3f50e06e5126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:29.063562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c6bef96a-01f7-4559-a4c6-3f50e06e5126 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:29.174938 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bef96a-01f7-4559-a4c6-3f50e06e5126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:47:29.176433 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bef96a-01f7-4559-a4c6-3f50e06e5126 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:47:29.195671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 660/1325] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:47:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:47:31.401639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d901750-f905-483f-b777-72ed6319c45b None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:31.402120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d901750-f905-483f-b777-72ed6319c45b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:31.402250 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d901750-f905-483f-b777-72ed6319c45b None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:31.403504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 666/1326] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:47:31.408854 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f42872-f73d-4dda-954a-5c49439ddfff None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:31.409263 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f42872-f73d-4dda-954a-5c49439ddfff None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:31.409394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f42872-f73d-4dda-954a-5c49439ddfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:31.458401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:31.477632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-01f42872-f73d-4dda-954a-5c49439ddfff None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:47:31.478826 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-01f42872-f73d-4dda-954a-5c49439ddfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:31.585439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 661/1327] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:47:31.865475 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d901750-f905-483f-b777-72ed6319c45b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:47:31.867008 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4d901750-f905-483f-b777-72ed6319c45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:31.969287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d901750-f905-483f-b777-72ed6319c45b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:47:32.040711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc462e3e-f373-48d4-ae0a-3bfbc2fe9bca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:47:32.041312 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc462e3e-f373-48d4-ae0a-3bfbc2fe9bca service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:32.041585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc462e3e-f373-48d4-ae0a-3bfbc2fe9bca service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:32.041849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc462e3e-f373-48d4-ae0a-3bfbc2fe9bca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:32.043418 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bc462e3e-f373-48d4-ae0a-3bfbc2fe9bca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:32.060215 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:32.152135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc462e3e-f373-48d4-ae0a-3bfbc2fe9bca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:47:32.153560 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc462e3e-f373-48d4-ae0a-3bfbc2fe9bca service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:47:32.171054 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 667/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:47:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:47:34.389132 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52498e1-9cd7-4e6a-879d-5a3b7b39e5d9 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:34.389528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52498e1-9cd7-4e6a-879d-5a3b7b39e5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:34.389672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52498e1-9cd7-4e6a-879d-5a3b7b39e5d9 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:34.391018 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 662/1329] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:47:34.397235 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e592055-1d75-41ea-8408-fd3405cddf08 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:34.397657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e592055-1d75-41ea-8408-fd3405cddf08 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:34.397781 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e592055-1d75-41ea-8408-fd3405cddf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:34.465658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1e592055-1d75-41ea-8408-fd3405cddf08 None None] MFA Rules not processed for user `3d9981f1b2c1421b8d5a89485cc87def`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:47:34.466808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1e592055-1d75-41ea-8408-fd3405cddf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:34.533789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:34.566176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 668/1330] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:47:34 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:47:34.850769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52498e1-9cd7-4e6a-879d-5a3b7b39e5d9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:47:34.852425 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c52498e1-9cd7-4e6a-879d-5a3b7b39e5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:34.965306 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52498e1-9cd7-4e6a-879d-5a3b7b39e5d9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:47:34.985459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:35.039321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df542cbe-01bd-45ed-b642-818b5961f551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:47:35.039932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df542cbe-01bd-45ed-b642-818b5961f551 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:35.040236 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df542cbe-01bd-45ed-b642-818b5961f551 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:35.040509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df542cbe-01bd-45ed-b642-818b5961f551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:35.042161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-df542cbe-01bd-45ed-b642-818b5961f551 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:35.166015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df542cbe-01bd-45ed-b642-818b5961f551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:47:35.167448 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df542cbe-01bd-45ed-b642-818b5961f551 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:47:35.186870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 663/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:47:34 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:47:37.758449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0698252-0e95-4898-912b-b89269caaa12 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:37.759051 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0698252-0e95-4898-912b-b89269caaa12 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:37.759051 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0698252-0e95-4898-912b-b89269caaa12 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:37.760278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 669/1332] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:47:37.765415 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a234c-1556-40a3-9c61-438f6ff2230a None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:37.765846 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a234c-1556-40a3-9c61-438f6ff2230a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:37.765948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a234c-1556-40a3-9c61-438f6ff2230a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:37.829074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:37.831937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a76a234c-1556-40a3-9c61-438f6ff2230a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:47:37.834212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a76a234c-1556-40a3-9c61-438f6ff2230a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:37.938819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 664/1333] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:47:38.219599 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0698252-0e95-4898-912b-b89269caaa12 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:47:38.221184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f0698252-0e95-4898-912b-b89269caaa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:38.324380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0698252-0e95-4898-912b-b89269caaa12 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:47:38.395315 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66a645-1162-418e-b158-b81c9a01be2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:47:38.395915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66a645-1162-418e-b158-b81c9a01be2c service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:38.396229 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66a645-1162-418e-b158-b81c9a01be2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:38.396500 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66a645-1162-418e-b158-b81c9a01be2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:38.398075 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2b66a645-1162-418e-b158-b81c9a01be2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:38.424632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:38.506966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66a645-1162-418e-b158-b81c9a01be2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:47:38.508398 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66a645-1162-418e-b158-b81c9a01be2c service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:47:38.526124 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 670/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:47:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:47:40.717448 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb107c-8b85-406a-affe-0353fe65e615 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:40.717448 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb107c-8b85-406a-affe-0353fe65e615 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:40.717448 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb107c-8b85-406a-affe-0353fe65e615 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:40.718819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 665/1335] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:47:40.725082 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb7e6a-2641-441a-aa45-dcca194ad878 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:40.725549 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb7e6a-2641-441a-aa45-dcca194ad878 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:40.725665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb7e6a-2641-441a-aa45-dcca194ad878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:40.790457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1deb7e6a-2641-441a-aa45-dcca194ad878 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:47:40.791659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1deb7e6a-2641-441a-aa45-dcca194ad878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:40.864259 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:40.887216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 671/1336] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:47:41.180913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb107c-8b85-406a-affe-0353fe65e615 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:47:41.182451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2bbb107c-8b85-406a-affe-0353fe65e615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:41.289970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb107c-8b85-406a-affe-0353fe65e615 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:47:41.321659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:41.370299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c64a0-14e1-4fc0-a7f3-03bd24cb28b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:47:41.370916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c64a0-14e1-4fc0-a7f3-03bd24cb28b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:41.371196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c64a0-14e1-4fc0-a7f3-03bd24cb28b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:41.371467 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c64a0-14e1-4fc0-a7f3-03bd24cb28b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:41.373089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7d9c64a0-14e1-4fc0-a7f3-03bd24cb28b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:41.486572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c64a0-14e1-4fc0-a7f3-03bd24cb28b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:47:41.488115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c64a0-14e1-4fc0-a7f3-03bd24cb28b5 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:47:41.506986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 666/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:47:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:47:43.967055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30d515-09cb-4df0-82e3-9e35372f7db9 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:43.967583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30d515-09cb-4df0-82e3-9e35372f7db9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:43.967583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30d515-09cb-4df0-82e3-9e35372f7db9 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:43.968980 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 672/1338] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:47:43.974729 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824b036-bbbc-4eb9-a002-56788170a416 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:43.975133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824b036-bbbc-4eb9-a002-56788170a416 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:43.975274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824b036-bbbc-4eb9-a002-56788170a416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:44.047018 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-0824b036-bbbc-4eb9-a002-56788170a416 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:47:44.048249 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0824b036-bbbc-4eb9-a002-56788170a416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:44.056699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:44.143012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 667/1339] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:47:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:47:44.429348 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30d515-09cb-4df0-82e3-9e35372f7db9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:47:44.430920 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9e30d515-09cb-4df0-82e3-9e35372f7db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:44.536176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30d515-09cb-4df0-82e3-9e35372f7db9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:47:44.609361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb2da4-a958-491f-81eb-7b8f5f2e4366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:47:44.609955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb2da4-a958-491f-81eb-7b8f5f2e4366 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:44.610236 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb2da4-a958-491f-81eb-7b8f5f2e4366 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:44.610505 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb2da4-a958-491f-81eb-7b8f5f2e4366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:44.612123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b9cb2da4-a958-491f-81eb-7b8f5f2e4366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:44.648572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:44.722330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb2da4-a958-491f-81eb-7b8f5f2e4366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:47:44.724052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb2da4-a958-491f-81eb-7b8f5f2e4366 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:47:44.743110 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 673/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:47:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:47:48.865589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36bf38-8d87-46ba-a78b-9242df360419 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:48.866112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36bf38-8d87-46ba-a78b-9242df360419 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:48.866112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36bf38-8d87-46ba-a78b-9242df360419 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:48.867476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 668/1341] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:47:48.873005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4049211-2022-46f3-9bbd-8eb507f46ef7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:48.873444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4049211-2022-46f3-9bbd-8eb507f46ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:48.873536 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4049211-2022-46f3-9bbd-8eb507f46ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:48.938311 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c4049211-2022-46f3-9bbd-8eb507f46ef7 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:47:48.939415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c4049211-2022-46f3-9bbd-8eb507f46ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:49.025737 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:49.038195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 674/1342] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:47:49.320647 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36bf38-8d87-46ba-a78b-9242df360419 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:47:49.322241 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-eb36bf38-8d87-46ba-a78b-9242df360419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:49.430979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36bf38-8d87-46ba-a78b-9242df360419 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:47:49.470644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:49.506679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0bf20-c2b0-4de7-a05f-627cc6df4c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:47:49.507685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0bf20-c2b0-4de7-a05f-627cc6df4c6c service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:49.508155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0bf20-c2b0-4de7-a05f-627cc6df4c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:49.508500 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0bf20-c2b0-4de7-a05f-627cc6df4c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:49.510133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dfa0bf20-c2b0-4de7-a05f-627cc6df4c6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:49.618786 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0bf20-c2b0-4de7-a05f-627cc6df4c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:47:49.620157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0bf20-c2b0-4de7-a05f-627cc6df4c6c service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:47:49.638297 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 669/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:47:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:47:52.542012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796e7fc-c13c-457e-a6d6-8d8400e4930f None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:52.542559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796e7fc-c13c-457e-a6d6-8d8400e4930f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:52.542559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796e7fc-c13c-457e-a6d6-8d8400e4930f None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:52.543897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 675/1344] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:47:52.549146 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1eead-df2e-4885-a3d4-a4f3bd990f5b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:52.549560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1eead-df2e-4885-a3d4-a4f3bd990f5b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:52.549677 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1eead-df2e-4885-a3d4-a4f3bd990f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:52.618856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-59a1eead-df2e-4885-a3d4-a4f3bd990f5b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:47:52.620067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-59a1eead-df2e-4885-a3d4-a4f3bd990f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:52.638703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:52.724938 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 670/1345] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:47:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:47:53.025122 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796e7fc-c13c-457e-a6d6-8d8400e4930f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:47:53.026666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5796e7fc-c13c-457e-a6d6-8d8400e4930f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:53.129115 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796e7fc-c13c-457e-a6d6-8d8400e4930f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:47:53.200671 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dcc8b2-eb9f-4937-9d2f-c5a9d74fa039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:47:53.201180 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcc8b2-eb9f-4937-9d2f-c5a9d74fa039 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:53.201452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcc8b2-eb9f-4937-9d2f-c5a9d74fa039 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:53.201721 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcc8b2-eb9f-4937-9d2f-c5a9d74fa039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:53.203288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-51dcc8b2-eb9f-4937-9d2f-c5a9d74fa039 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:53.250217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:53.319161 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dcc8b2-eb9f-4937-9d2f-c5a9d74fa039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:47:53.320620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dcc8b2-eb9f-4937-9d2f-c5a9d74fa039 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:47:53.338828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 676/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:47:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:47:57.516650 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eba5a4-0f7c-4578-ae65-d74ee1e7165e None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:57.517080 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eba5a4-0f7c-4578-ae65-d74ee1e7165e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:57.517301 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eba5a4-0f7c-4578-ae65-d74ee1e7165e None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:57.519055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 671/1347] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:47:57.524691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b16605-4124-47f2-ab33-f93ec602db4a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:57.525079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b16605-4124-47f2-ab33-f93ec602db4a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:57.525293 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b16605-4124-47f2-ab33-f93ec602db4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:57.594018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-72b16605-4124-47f2-ab33-f93ec602db4a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:47:57.595250 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-72b16605-4124-47f2-ab33-f93ec602db4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:57.704964 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:57.717042 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 677/1348] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:47:57.994413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eba5a4-0f7c-4578-ae65-d74ee1e7165e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:47:57.995994 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-92eba5a4-0f7c-4578-ae65-d74ee1e7165e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:58.099226 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eba5a4-0f7c-4578-ae65-d74ee1e7165e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:47:58.146125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:47:58.171852 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487f4e0-de6d-4bd9-9ee4-53bc34046bff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:47:58.172487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487f4e0-de6d-4bd9-9ee4-53bc34046bff service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:47:58.172771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487f4e0-de6d-4bd9-9ee4-53bc34046bff service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:47:58.173041 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487f4e0-de6d-4bd9-9ee4-53bc34046bff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:47:58.174609 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6487f4e0-de6d-4bd9-9ee4-53bc34046bff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:47:58.281944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487f4e0-de6d-4bd9-9ee4-53bc34046bff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:47:58.283293 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487f4e0-de6d-4bd9-9ee4-53bc34046bff service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:47:58.301256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 672/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:47:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:48:03.300738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc64c79-da38-4346-8810-79ccb7454731 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:03.301164 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc64c79-da38-4346-8810-79ccb7454731 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:03.301257 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc64c79-da38-4346-8810-79ccb7454731 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:03.302611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 678/1350] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 7 07:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:03.307523 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c995b6-16f8-49de-b610-edbbb32122df None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:03.307953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c995b6-16f8-49de-b610-edbbb32122df None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:03.308292 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c995b6-16f8-49de-b610-edbbb32122df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:03.426784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c8c995b6-16f8-49de-b610-edbbb32122df None None] MFA Rules not processed for user `b16b81ab30294bc4a8c87b6d7c3573f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:03.427936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c8c995b6-16f8-49de-b610-edbbb32122df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:03.460520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:03.571180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 673/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 7 07:48:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:07.619976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44d6d1-d0aa-489e-91b4-ff83ebe84aed None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:07.620562 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44d6d1-d0aa-489e-91b4-ff83ebe84aed None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:07.620562 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44d6d1-d0aa-489e-91b4-ff83ebe84aed None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:07.622045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 679/1352] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 7 07:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:07.626826 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2238a-f243-4f24-acbe-e0e2f4fc96b2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:07.628168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2238a-f243-4f24-acbe-e0e2f4fc96b2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:07.628666 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2238a-f243-4f24-acbe-e0e2f4fc96b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:07.715650 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-93b2238a-f243-4f24-acbe-e0e2f4fc96b2 None None] MFA Rules not processed for user `b16b81ab30294bc4a8c87b6d7c3573f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:07.717212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-93b2238a-f243-4f24-acbe-e0e2f4fc96b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:07.796372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:07.841049 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 674/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 7 07:48:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:09.390913 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd3d93-e874-4e71-bde2-a3727625caa9 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:09.391367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd3d93-e874-4e71-bde2-a3727625caa9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:09.391588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd3d93-e874-4e71-bde2-a3727625caa9 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:09.393580 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 680/1354] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 7 07:48:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:09.399052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c79b4-273c-489a-98be-27fd6fc62ffb None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:09.399492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c79b4-273c-489a-98be-27fd6fc62ffb None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:09.399606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c79b4-273c-489a-98be-27fd6fc62ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:09.477800 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-6b2c79b4-273c-489a-98be-27fd6fc62ffb None None] MFA Rules not processed for user `b16b81ab30294bc4a8c87b6d7c3573f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:09.478990 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6b2c79b4-273c-489a-98be-27fd6fc62ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:09.588449 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 675/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 7 07:48:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:13.186159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833da86-63bb-430f-b422-97b882ba8b49 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:13.186693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833da86-63bb-430f-b422-97b882ba8b49 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:13.187384 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833da86-63bb-430f-b422-97b882ba8b49 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:13.189017 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 681/1356] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:48:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:13.194989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7f340-ca89-417f-a72d-332c56280f98 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:13.195798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7f340-ca89-417f-a72d-332c56280f98 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:13.196391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7f340-ca89-417f-a72d-332c56280f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:13.239170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:13.275584 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-8be7f340-ca89-417f-a72d-332c56280f98 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:13.276812 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8be7f340-ca89-417f-a72d-332c56280f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:13.388218 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 676/1357] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:48:13.693111 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24df7d-94b6-454d-9072-96d68609e8a4 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:13.693687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24df7d-94b6-454d-9072-96d68609e8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:13.694332 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24df7d-94b6-454d-9072-96d68609e8a4 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:13.695971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 682/1358] 10.222.0.26 () {54 vars in 962 bytes} [Wed Sep 7 07:48:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:13.704891 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676417de-6e03-4fd0-94b7-f32cfbdf32c5 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:13.705389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676417de-6e03-4fd0-94b7-f32cfbdf32c5 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:13.705915 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676417de-6e03-4fd0-94b7-f32cfbdf32c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:13.788503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-676417de-6e03-4fd0-94b7-f32cfbdf32c5 None None] MFA Rules not processed for user `81d5689e9a344bafb74ca423ffd2b42a`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:13.789699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-676417de-6e03-4fd0-94b7-f32cfbdf32c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:13.841947 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:13.915537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 677/1359] 10.222.0.26 () {58 vars in 1059 bytes} [Wed Sep 7 07:48:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:48:13.928103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24df7d-94b6-454d-9072-96d68609e8a4 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:48:13.930028 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-de24df7d-94b6-454d-9072-96d68609e8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:14.114567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24df7d-94b6-454d-9072-96d68609e8a4 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:48:14.188918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:14.229528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c14bc-cbda-45a1-859b-d3c553609551 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:48:14.230120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c14bc-cbda-45a1-859b-d3c553609551 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:14.230393 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c14bc-cbda-45a1-859b-d3c553609551 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:14.230661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c14bc-cbda-45a1-859b-d3c553609551 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:14.232266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ea7c14bc-cbda-45a1-859b-d3c553609551 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:14.329548 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c14bc-cbda-45a1-859b-d3c553609551 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:48:14.331913 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c14bc-cbda-45a1-859b-d3c553609551 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:48:14.352207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 683/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:48:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:48:16.753652 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246d72a-b1bf-4e5a-9421-531117df8b8d None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:16.754176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246d72a-b1bf-4e5a-9421-531117df8b8d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:16.754301 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246d72a-b1bf-4e5a-9421-531117df8b8d None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:16.755557 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 678/1361] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:16.761063 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334255ae-4c4d-46c5-a895-7351883ed27b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:16.761486 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334255ae-4c4d-46c5-a895-7351883ed27b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:16.761594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334255ae-4c4d-46c5-a895-7351883ed27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:16.865320 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-334255ae-4c4d-46c5-a895-7351883ed27b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:16.866533 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-334255ae-4c4d-46c5-a895-7351883ed27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:16.906506 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:16.976231 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 684/1362] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:17.266229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246d72a-b1bf-4e5a-9421-531117df8b8d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:48:17.267921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9246d72a-b1bf-4e5a-9421-531117df8b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:17.437915 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246d72a-b1bf-4e5a-9421-531117df8b8d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:48:17.459266 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:17.577967 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc992f28-88dd-4863-9c31-cf5098304623 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:48:17.578573 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc992f28-88dd-4863-9c31-cf5098304623 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:17.578856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc992f28-88dd-4863-9c31-cf5098304623 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:17.579126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc992f28-88dd-4863-9c31-cf5098304623 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:17.581111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dc992f28-88dd-4863-9c31-cf5098304623 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:17.675042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc992f28-88dd-4863-9c31-cf5098304623 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:48:17.676798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc992f28-88dd-4863-9c31-cf5098304623 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:48:17.695064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 679/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:48:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:48:20.208169 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c648537-150b-4836-8408-06496ef6c80a None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:20.208733 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c648537-150b-4836-8408-06496ef6c80a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:20.208733 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c648537-150b-4836-8408-06496ef6c80a None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:20.210226 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 685/1364] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:20.215520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f68d6-8237-49a1-afc6-76b3055d084f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:20.215924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f68d6-8237-49a1-afc6-76b3055d084f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:20.216129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f68d6-8237-49a1-afc6-76b3055d084f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:20.232329 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:20.288212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f03f68d6-8237-49a1-afc6-76b3055d084f None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:20.289429 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f03f68d6-8237-49a1-afc6-76b3055d084f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:20.394286 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 680/1365] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:20.691584 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1c4fc-7921-478c-b771-cdf501d79d1a None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:20.692040 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1c4fc-7921-478c-b771-cdf501d79d1a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:20.692434 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1c4fc-7921-478c-b771-cdf501d79d1a None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:20.693942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 686/1366] 10.222.0.26 () {54 vars in 962 bytes} [Wed Sep 7 07:48:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:20.699263 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f26012-329f-4aef-aebf-283d3ec2c9b5 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:20.699871 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f26012-329f-4aef-aebf-283d3ec2c9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:20.700232 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f26012-329f-4aef-aebf-283d3ec2c9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:20.779371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-20f26012-329f-4aef-aebf-283d3ec2c9b5 None None] MFA Rules not processed for user `81d5689e9a344bafb74ca423ffd2b42a`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:20.780609 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-20f26012-329f-4aef-aebf-283d3ec2c9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:20.817722 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:20.904813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 681/1367] 10.222.0.26 () {58 vars in 1059 bytes} [Wed Sep 7 07:48:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:20.913357 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1c4fc-7921-478c-b771-cdf501d79d1a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:48:20.915154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-44f1c4fc-7921-478c-b771-cdf501d79d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:21.073439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1c4fc-7921-478c-b771-cdf501d79d1a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:48:21.095019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:21.196596 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c76834-82c4-41bf-9555-f51b0a5b5d4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:48:21.197206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c76834-82c4-41bf-9555-f51b0a5b5d4e service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:21.197489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c76834-82c4-41bf-9555-f51b0a5b5d4e service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:21.197760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c76834-82c4-41bf-9555-f51b0a5b5d4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:21.199360 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f6c76834-82c4-41bf-9555-f51b0a5b5d4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:21.295309 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c76834-82c4-41bf-9555-f51b0a5b5d4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:48:21.296956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c76834-82c4-41bf-9555-f51b0a5b5d4e service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:48:21.316831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 687/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:48:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:48:21.500679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42402859-8d87-4e01-88ed-8b81e1ffd530 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:21.501266 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42402859-8d87-4e01-88ed-8b81e1ffd530 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:21.501434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42402859-8d87-4e01-88ed-8b81e1ffd530 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:21.503930 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 682/1369] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:48:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:21.510245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4238e9-8c35-48a0-9b7c-047527c9533b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:21.510658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4238e9-8c35-48a0-9b7c-047527c9533b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:21.510829 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4238e9-8c35-48a0-9b7c-047527c9533b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:21.553474 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:21.598323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ce4238e9-8c35-48a0-9b7c-047527c9533b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:21.599568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ce4238e9-8c35-48a0-9b7c-047527c9533b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:21.705717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 688/1370] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:22.040165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479119e-7206-442c-844a-00cdd78685c3 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:22.040165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479119e-7206-442c-844a-00cdd78685c3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:22.040165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479119e-7206-442c-844a-00cdd78685c3 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:22.041174 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 683/1371] 10.222.0.26 () {54 vars in 964 bytes} [Wed Sep 7 07:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:22.045644 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba46424-1761-4045-b18c-ccb378a1fc1b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:22.045934 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba46424-1761-4045-b18c-ccb378a1fc1b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:22.046212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba46424-1761-4045-b18c-ccb378a1fc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:22.130820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9ba46424-1761-4045-b18c-ccb378a1fc1b None None] MFA Rules not processed for user `92699b512c064b05948afe597650fdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:22.132175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9ba46424-1761-4045-b18c-ccb378a1fc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:22.180970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:22.214562 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 689/1372] 10.222.0.26 () {58 vars in 1061 bytes} [Wed Sep 7 07:48:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:22.222657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479119e-7206-442c-844a-00cdd78685c3 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:48:22.224252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3479119e-7206-442c-844a-00cdd78685c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:22.351083 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479119e-7206-442c-844a-00cdd78685c3 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:48:22.396427 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:22.439041 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05f11d-2216-47cd-b4ba-fd8d2fb9ab3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:48:22.440347 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05f11d-2216-47cd-b4ba-fd8d2fb9ab3e service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:22.440762 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05f11d-2216-47cd-b4ba-fd8d2fb9ab3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:22.441147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05f11d-2216-47cd-b4ba-fd8d2fb9ab3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:22.442707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1c05f11d-2216-47cd-b4ba-fd8d2fb9ab3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:22.561173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05f11d-2216-47cd-b4ba-fd8d2fb9ab3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:48:22.563016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05f11d-2216-47cd-b4ba-fd8d2fb9ab3e service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:48:22.582766 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 684/1373] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:48:23.608988 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570f380-487b-4d2e-b36a-e1a377e5f175 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:23.609496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570f380-487b-4d2e-b36a-e1a377e5f175 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:23.609496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570f380-487b-4d2e-b36a-e1a377e5f175 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:23.610881 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 690/1374] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:23.616979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fcbef-bc27-410e-b508-31d4b73cba8d None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:23.617383 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fcbef-bc27-410e-b508-31d4b73cba8d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:23.617531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fcbef-bc27-410e-b508-31d4b73cba8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:23.683220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-003fcbef-bc27-410e-b508-31d4b73cba8d None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:23.684457 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-003fcbef-bc27-410e-b508-31d4b73cba8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:23.705676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:23.782308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 685/1375] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:24.078366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570f380-487b-4d2e-b36a-e1a377e5f175 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:48:24.080764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e570f380-487b-4d2e-b36a-e1a377e5f175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:24.245025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570f380-487b-4d2e-b36a-e1a377e5f175 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:48:24.299012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:24.363866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62297ada-265d-4812-a73e-8716a4a41c8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:48:24.364598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62297ada-265d-4812-a73e-8716a4a41c8e service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:24.364877 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62297ada-265d-4812-a73e-8716a4a41c8e service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:24.365146 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62297ada-265d-4812-a73e-8716a4a41c8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:24.366755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-62297ada-265d-4812-a73e-8716a4a41c8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:24.458751 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62297ada-265d-4812-a73e-8716a4a41c8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:48:24.460371 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62297ada-265d-4812-a73e-8716a4a41c8e service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:48:24.479260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 691/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:48:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:48:26.711415 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75979567-9a52-4fcc-8f78-f65c697069bd None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:26.711908 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75979567-9a52-4fcc-8f78-f65c697069bd None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:26.711908 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75979567-9a52-4fcc-8f78-f65c697069bd None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:26.713370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 686/1377] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:26.718403 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0879b5-23b7-43b3-bd2a-35ddbc2facb5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:26.718782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0879b5-23b7-43b3-bd2a-35ddbc2facb5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:26.718944 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0879b5-23b7-43b3-bd2a-35ddbc2facb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:26.786327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5c0879b5-23b7-43b3-bd2a-35ddbc2facb5 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:26.787519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5c0879b5-23b7-43b3-bd2a-35ddbc2facb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:26.794645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:26.893587 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 692/1378] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:27.183587 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75979567-9a52-4fcc-8f78-f65c697069bd None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:48:27.185275 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-75979567-9a52-4fcc-8f78-f65c697069bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:27.341503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75979567-9a52-4fcc-8f78-f65c697069bd None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:48:27.346521 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:27.475178 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b66364-7b54-4082-b19b-f72ac7cbb966 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:48:27.475891 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b66364-7b54-4082-b19b-f72ac7cbb966 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:27.476299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b66364-7b54-4082-b19b-f72ac7cbb966 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:27.476753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b66364-7b54-4082-b19b-f72ac7cbb966 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:27.478381 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e5b66364-7b54-4082-b19b-f72ac7cbb966 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:27.578375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b66364-7b54-4082-b19b-f72ac7cbb966 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:48:27.580543 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b66364-7b54-4082-b19b-f72ac7cbb966 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:48:27.601094 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 687/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:48:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:48:29.824661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004e586-8ee5-4908-a661-60cd14b8ab39 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:29.825158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004e586-8ee5-4908-a661-60cd14b8ab39 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:29.825158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004e586-8ee5-4908-a661-60cd14b8ab39 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:29.826612 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 693/1380] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:29.831657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b6239-2c9c-4d75-ace4-11cba0373891 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:29.832057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b6239-2c9c-4d75-ace4-11cba0373891 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:29.832269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b6239-2c9c-4d75-ace4-11cba0373891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:29.843664 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:29.900676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7f6b6239-2c9c-4d75-ace4-11cba0373891 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:29.901886 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7f6b6239-2c9c-4d75-ace4-11cba0373891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:30.009764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 688/1381] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:30.316511 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004e586-8ee5-4908-a661-60cd14b8ab39 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:48:30.318410 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f004e586-8ee5-4908-a661-60cd14b8ab39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:30.464905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:30.477397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004e586-8ee5-4908-a661-60cd14b8ab39 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:48:30.611124 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc21b5-b8ea-43ba-a4a7-83fbf3cd2d2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:48:30.611727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc21b5-b8ea-43ba-a4a7-83fbf3cd2d2b service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:30.612004 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc21b5-b8ea-43ba-a4a7-83fbf3cd2d2b service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:30.612303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc21b5-b8ea-43ba-a4a7-83fbf3cd2d2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:30.613900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-81bc21b5-b8ea-43ba-a4a7-83fbf3cd2d2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:30.705549 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc21b5-b8ea-43ba-a4a7-83fbf3cd2d2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:48:30.707007 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc21b5-b8ea-43ba-a4a7-83fbf3cd2d2b service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:48:30.716839 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:30.725683 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 694/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:48:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:48:33.506704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b75a98-bf2c-462c-b68e-b1ccfb4f5f6d None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:33.507273 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b75a98-bf2c-462c-b68e-b1ccfb4f5f6d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:33.507401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b75a98-bf2c-462c-b68e-b1ccfb4f5f6d None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:33.509256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 689/1383] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:48:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:33.514888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135a524-a491-4087-9528-41b0751d4c24 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:33.515312 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135a524-a491-4087-9528-41b0751d4c24 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:33.515456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135a524-a491-4087-9528-41b0751d4c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:33.596591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7135a524-a491-4087-9528-41b0751d4c24 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:33.598085 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7135a524-a491-4087-9528-41b0751d4c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:33.704617 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 695/1384] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:34.007649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b75a98-bf2c-462c-b68e-b1ccfb4f5f6d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:48:34.009244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f9b75a98-bf2c-462c-b68e-b1ccfb4f5f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:34.096883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:34.161055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b75a98-bf2c-462c-b68e-b1ccfb4f5f6d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:48:34.301396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fb394-7ed0-4212-8731-f57313c28a48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:48:34.302063 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fb394-7ed0-4212-8731-f57313c28a48 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:34.302490 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fb394-7ed0-4212-8731-f57313c28a48 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:34.302826 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fb394-7ed0-4212-8731-f57313c28a48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:34.304913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-986fb394-7ed0-4212-8731-f57313c28a48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:34.357762 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:34.399338 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fb394-7ed0-4212-8731-f57313c28a48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:48:34.400962 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fb394-7ed0-4212-8731-f57313c28a48 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:48:34.420143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 690/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:48:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:48:36.853367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd4df2-36aa-41b7-93c4-608c1e911a74 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:36.854656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd4df2-36aa-41b7-93c4-608c1e911a74 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:36.855773 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd4df2-36aa-41b7-93c4-608c1e911a74 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:36.857258 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 696/1386] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:48:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:36.864272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d593e-936b-4981-99a2-2359d6a058b7 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:36.864733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d593e-936b-4981-99a2-2359d6a058b7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:36.864955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d593e-936b-4981-99a2-2359d6a058b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:36.933213 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-fd0d593e-936b-4981-99a2-2359d6a058b7 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:36.934409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fd0d593e-936b-4981-99a2-2359d6a058b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:37.031868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 691/1387] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:48:37.320525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd4df2-36aa-41b7-93c4-608c1e911a74 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:48:37.322047 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cdcd4df2-36aa-41b7-93c4-608c1e911a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:37.400953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:37.473903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd4df2-36aa-41b7-93c4-608c1e911a74 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:48:37.594001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8156d2-8b43-4e3c-be63-59ed88cf7700 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:48:37.594673 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8156d2-8b43-4e3c-be63-59ed88cf7700 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:37.594988 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8156d2-8b43-4e3c-be63-59ed88cf7700 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:37.595339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8156d2-8b43-4e3c-be63-59ed88cf7700 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:37.597040 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0c8156d2-8b43-4e3c-be63-59ed88cf7700 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:37.635084 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:37.693752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8156d2-8b43-4e3c-be63-59ed88cf7700 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:48:37.695341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8156d2-8b43-4e3c-be63-59ed88cf7700 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:48:37.713664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 697/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:48:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:48:37.942550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27424e-409f-4995-bd8b-1176c24beb6a None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:37.943115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27424e-409f-4995-bd8b-1176c24beb6a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:37.943115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27424e-409f-4995-bd8b-1176c24beb6a None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:37.944491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 692/1389] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:37.950027 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55612e5f-bb1b-46f6-bcd7-b52552a958ea None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:37.950462 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55612e5f-bb1b-46f6-bcd7-b52552a958ea None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:37.950584 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55612e5f-bb1b-46f6-bcd7-b52552a958ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:38.017556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-55612e5f-bb1b-46f6-bcd7-b52552a958ea None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:38.018822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-55612e5f-bb1b-46f6-bcd7-b52552a958ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:38.114904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 698/1390] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:38.413512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d7815-e91a-4b5c-96b4-8c79cbc2ad07 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:38.413876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d7815-e91a-4b5c-96b4-8c79cbc2ad07 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:38.414224 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d7815-e91a-4b5c-96b4-8c79cbc2ad07 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:38.415773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 693/1391] 10.222.0.26 () {54 vars in 961 bytes} [Wed Sep 7 07:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:38.421648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff7ad9-4e6e-4305-9a73-53a418643680 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:38.421937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff7ad9-4e6e-4305-9a73-53a418643680 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:38.422210 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff7ad9-4e6e-4305-9a73-53a418643680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:38.437798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:38.500366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-bfff7ad9-4e6e-4305-9a73-53a418643680 None None] MFA Rules not processed for user `9b0652c89c9945e2866512c9da4793ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:38.502112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bfff7ad9-4e6e-4305-9a73-53a418643680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:38.577294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 699/1392] 10.222.0.26 () {58 vars in 1058 bytes} [Wed Sep 7 07:48:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:38.586962 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d7815-e91a-4b5c-96b4-8c79cbc2ad07 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:48:38.588516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3a3d7815-e91a-4b5c-96b4-8c79cbc2ad07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:38.619938 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:38.707051 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d7815-e91a-4b5c-96b4-8c79cbc2ad07 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:48:38.783173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a70e9d-f0e6-4094-b232-5578a8c91112 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b0652c89c9945e2866512c9da4793ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:48:38.783773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a70e9d-f0e6-4094-b232-5578a8c91112 service glance] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:38.784045 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a70e9d-f0e6-4094-b232-5578a8c91112 service glance] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:38.784340 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a70e9d-f0e6-4094-b232-5578a8c91112 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:38.785912 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-98a70e9d-f0e6-4094-b232-5578a8c91112 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:38.866929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:38.901700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a70e9d-f0e6-4094-b232-5578a8c91112 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:48:38.903168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a70e9d-f0e6-4094-b232-5578a8c91112 service glance] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:48:38.921588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 694/1393] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:48:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:48:39.683098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a0352-d77a-452b-87ed-2e975ff7d8af None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:39.683554 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a0352-d77a-452b-87ed-2e975ff7d8af None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:39.683781 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a0352-d77a-452b-87ed-2e975ff7d8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:39.751541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1c8a0352-d77a-452b-87ed-2e975ff7d8af None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:39.752868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1c8a0352-d77a-452b-87ed-2e975ff7d8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:39.808642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:39.848211 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 700/1394] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Sep 7 07:48:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:39.894737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a93d2-5fc4-43d6-9b0f-21bc5e18dc9c None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:39.895226 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a93d2-5fc4-43d6-9b0f-21bc5e18dc9c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:39.895323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a93d2-5fc4-43d6-9b0f-21bc5e18dc9c None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:39.896719 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 695/1395] 10.222.0.26 () {54 vars in 943 bytes} [Wed Sep 7 07:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:39.902984 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d8d80-613b-400c-a88c-5afd3d8ab68a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:39.903432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d8d80-613b-400c-a88c-5afd3d8ab68a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:39.903702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d8d80-613b-400c-a88c-5afd3d8ab68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:39.934828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3a322-6548-44a3-993c-b845515c2a8b None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:39.935030 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3a322-6548-44a3-993c-b845515c2a8b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:39.935286 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3a322-6548-44a3-993c-b845515c2a8b None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:39.936756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 696/1396] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:48:39.942727 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ad07d-1603-4b2a-95c4-cacbecf750a0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:39.943005 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ad07d-1603-4b2a-95c4-cacbecf750a0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:39.943278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ad07d-1603-4b2a-95c4-cacbecf750a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:39.982212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c44d8d80-613b-400c-a88c-5afd3d8ab68a None None] MFA Rules not processed for user `3ff3bf8da732489b8b3a8fbe5aa22f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:39.983420 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c44d8d80-613b-400c-a88c-5afd3d8ab68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:40.011844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1f5ad07d-1603-4b2a-95c4-cacbecf750a0 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:40.013099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1f5ad07d-1603-4b2a-95c4-cacbecf750a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:40.093924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 701/1397] 10.222.0.26 () {58 vars in 1040 bytes} [Wed Sep 7 07:48:39 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:48:40.103397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d8d80-613b-400c-a88c-5afd3d8ab68a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:48:40.113182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d8d80-613b-400c-a88c-5afd3d8ab68a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:48:40.114626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 697/1398] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:48:40.138385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:40.216906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b3e07-d885-4c61-bf53-9089fd83bdd4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:48:40.217426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b3e07-d885-4c61-bf53-9089fd83bdd4 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:40.217700 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b3e07-d885-4c61-bf53-9089fd83bdd4 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:40.217968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b3e07-d885-4c61-bf53-9089fd83bdd4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:40.219605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-047b3e07-d885-4c61-bf53-9089fd83bdd4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:40.301138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b3e07-d885-4c61-bf53-9089fd83bdd4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:48:40.302708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b3e07-d885-4c61-bf53-9089fd83bdd4 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:48:40.310690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 702/1399] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 07:48:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 07:48:40.409590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ad07d-1603-4b2a-95c4-cacbecf750a0 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:48:40.410811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1f5ad07d-1603-4b2a-95c4-cacbecf750a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:40.459974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:40.545242 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ad07d-1603-4b2a-95c4-cacbecf750a0 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:48:40.676407 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369066b-7394-4b53-8c9f-d6955104e981 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:48:40.677465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369066b-7394-4b53-8c9f-d6955104e981 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:40.677823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369066b-7394-4b53-8c9f-d6955104e981 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:40.678168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369066b-7394-4b53-8c9f-d6955104e981 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:40.679875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2369066b-7394-4b53-8c9f-d6955104e981 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:40.724644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:40.789228 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2369066b-7394-4b53-8c9f-d6955104e981 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:48:40.791462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2369066b-7394-4b53-8c9f-d6955104e981 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:48:40.814502 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 698/1400] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:48:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:48:43.102485 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452c8e7-5dd3-4a88-b6c3-9136157da676 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:43.103044 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452c8e7-5dd3-4a88-b6c3-9136157da676 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:43.103044 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452c8e7-5dd3-4a88-b6c3-9136157da676 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:43.104519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 703/1401] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:43.109771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefa88a-2d4f-4478-9731-6d9edbb3a403 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:43.110817 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefa88a-2d4f-4478-9731-6d9edbb3a403 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:43.110920 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefa88a-2d4f-4478-9731-6d9edbb3a403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:43.185651 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-8aefa88a-2d4f-4478-9731-6d9edbb3a403 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:43.186865 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8aefa88a-2d4f-4478-9731-6d9edbb3a403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:43.292855 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 699/1402] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:48:43.597926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452c8e7-5dd3-4a88-b6c3-9136157da676 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:48:43.599472 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9452c8e7-5dd3-4a88-b6c3-9136157da676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:43.672171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:43.768536 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452c8e7-5dd3-4a88-b6c3-9136157da676 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:48:43.895665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3f8b8-3eca-463f-a19e-1e9c2b8cf3be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:48:43.896616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3f8b8-3eca-463f-a19e-1e9c2b8cf3be service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:43.897116 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3f8b8-3eca-463f-a19e-1e9c2b8cf3be service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:43.897555 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3f8b8-3eca-463f-a19e-1e9c2b8cf3be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:43.899268 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-48b3f8b8-3eca-463f-a19e-1e9c2b8cf3be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:43.918786 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:44.003962 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3f8b8-3eca-463f-a19e-1e9c2b8cf3be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:48:44.005725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3f8b8-3eca-463f-a19e-1e9c2b8cf3be service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:48:44.025861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 704/1403] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:48:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:48:46.256033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfb4da-0408-416c-9f1a-8db7f4405883 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:46.256632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfb4da-0408-416c-9f1a-8db7f4405883 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:46.256632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfb4da-0408-416c-9f1a-8db7f4405883 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:46.260248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 700/1404] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:48:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:46.263618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0361bdb-63db-44ef-8cf4-443162cdc93a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:46.264069 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0361bdb-63db-44ef-8cf4-443162cdc93a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:46.264192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0361bdb-63db-44ef-8cf4-443162cdc93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:46.335249 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d0361bdb-63db-44ef-8cf4-443162cdc93a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:46.336495 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d0361bdb-63db-44ef-8cf4-443162cdc93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:46.445665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:46.456000 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 705/1405] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:46.744347 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfb4da-0408-416c-9f1a-8db7f4405883 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:48:46.746090 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-62bfb4da-0408-416c-9f1a-8db7f4405883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:46.760031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:46.924442 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfb4da-0408-416c-9f1a-8db7f4405883 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:48:47.022525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:47.057793 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf7cd7-cf10-4622-9471-6c839185e96e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:48:47.058397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf7cd7-cf10-4622-9471-6c839185e96e service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:47.058675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf7cd7-cf10-4622-9471-6c839185e96e service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:47.058946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf7cd7-cf10-4622-9471-6c839185e96e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:47.060584 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-86cf7cd7-cf10-4622-9471-6c839185e96e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:47.184810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf7cd7-cf10-4622-9471-6c839185e96e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:48:47.186508 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf7cd7-cf10-4622-9471-6c839185e96e service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:48:47.215314 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 701/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:48:49.441498 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973d770-ad38-4c3d-ab1e-1e2f0023a3fe None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:49.442048 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973d770-ad38-4c3d-ab1e-1e2f0023a3fe None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:49.442048 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973d770-ad38-4c3d-ab1e-1e2f0023a3fe None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:49.443650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 706/1407] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:48:49.448722 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20f7d4-edc3-4979-8987-4329691a6f15 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:49.449141 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20f7d4-edc3-4979-8987-4329691a6f15 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:49.449290 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20f7d4-edc3-4979-8987-4329691a6f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:49.528073 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ce20f7d4-edc3-4979-8987-4329691a6f15 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:48:49.529491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ce20f7d4-edc3-4979-8987-4329691a6f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:49.587498 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:49.650484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 702/1408] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:48:49.943666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973d770-ad38-4c3d-ab1e-1e2f0023a3fe None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:48:49.945260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6973d770-ad38-4c3d-ab1e-1e2f0023a3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:50.114628 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973d770-ad38-4c3d-ab1e-1e2f0023a3fe None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:48:50.187437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:48:50.238736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533076c9-e6b5-4c20-86eb-bbf1d7c0d9bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:48:50.239335 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533076c9-e6b5-4c20-86eb-bbf1d7c0d9bc service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:48:50.239605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533076c9-e6b5-4c20-86eb-bbf1d7c0d9bc service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:48:50.239869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533076c9-e6b5-4c20-86eb-bbf1d7c0d9bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:48:50.241497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-533076c9-e6b5-4c20-86eb-bbf1d7c0d9bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:48:50.343586 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533076c9-e6b5-4c20-86eb-bbf1d7c0d9bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:48:50.345761 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533076c9-e6b5-4c20-86eb-bbf1d7c0d9bc service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:48:50.368602 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 707/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:48:59.648456 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=23 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:00.366077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=64 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:03.044909 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2815ec-a14e-4fa8-ac3f-6472e1966a7f None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:03.045491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2815ec-a14e-4fa8-ac3f-6472e1966a7f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:03.045752 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2815ec-a14e-4fa8-ac3f-6472e1966a7f None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:03.047133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 703/1410] 10.222.0.26 () {54 vars in 918 bytes} [Wed Sep 7 07:49:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:03.053116 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef7994-5f0a-402e-916f-29715a4dcf76 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:03.053525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef7994-5f0a-402e-916f-29715a4dcf76 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:03.053655 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef7994-5f0a-402e-916f-29715a4dcf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:03.132861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cdef7994-5f0a-402e-916f-29715a4dcf76 None None] MFA Rules not processed for user `b16b81ab30294bc4a8c87b6d7c3573f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:03.134908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cdef7994-5f0a-402e-916f-29715a4dcf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:03.256436 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 708/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Sep 7 07:49:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:03.271282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaae706-7813-4a3d-ab52-5f4a628963cd None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:03.271569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaae706-7813-4a3d-ab52-5f4a628963cd None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:03.271846 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaae706-7813-4a3d-ab52-5f4a628963cd None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:03.273213 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 704/1412] 10.222.0.26 () {54 vars in 918 bytes} [Wed Sep 7 07:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:49:03.278113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e9440-5be4-4357-90f7-f1958e8c2bf2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:03.278392 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e9440-5be4-4357-90f7-f1958e8c2bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:03.278663 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e9440-5be4-4357-90f7-f1958e8c2bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:03.362824 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:03.365739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5c6e9440-5be4-4357-90f7-f1958e8c2bf2 None None] MFA Rules not processed for user `81d5689e9a344bafb74ca423ffd2b42a`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:03.367574 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5c6e9440-5be4-4357-90f7-f1958e8c2bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:03.501384 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 709/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Sep 7 07:49:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:06.181598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560b567-7ca6-4578-9449-2dc1da531901 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:06.182133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560b567-7ca6-4578-9449-2dc1da531901 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:06.182133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560b567-7ca6-4578-9449-2dc1da531901 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:06.183482 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 07:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:06.189232 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c444a-012b-4222-8239-eb0cbae6e9f1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:06.189676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c444a-012b-4222-8239-eb0cbae6e9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:06.189789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c444a-012b-4222-8239-eb0cbae6e9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:06.256445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-464c444a-012b-4222-8239-eb0cbae6e9f1 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:06.257642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-464c444a-012b-4222-8239-eb0cbae6e9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:06.311663 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:06.353344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 07:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:06.646730 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560b567-7ca6-4578-9449-2dc1da531901 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:06.648285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c560b567-7ca6-4578-9449-2dc1da531901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:06.791858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560b567-7ca6-4578-9449-2dc1da531901 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:06.865162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:06.904893 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c50f0b-2afe-440f-af3a-56cdd9322304 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:06.905428 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c50f0b-2afe-440f-af3a-56cdd9322304 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:06.905705 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c50f0b-2afe-440f-af3a-56cdd9322304 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:06.905974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c50f0b-2afe-440f-af3a-56cdd9322304 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:06.908209 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-26c50f0b-2afe-440f-af3a-56cdd9322304 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:06.996008 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c50f0b-2afe-440f-af3a-56cdd9322304 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:06.997625 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c50f0b-2afe-440f-af3a-56cdd9322304 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:07.015593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 706/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:08.171994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4346738-357f-4774-b8fa-18975fa225ef None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:08.173168 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4346738-357f-4774-b8fa-18975fa225ef None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:08.173168 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4346738-357f-4774-b8fa-18975fa225ef None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:08.174479 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 07:49:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:08.180655 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0582f-2c55-43e9-8907-268c67645358 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:08.181056 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0582f-2c55-43e9-8907-268c67645358 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:08.181214 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0582f-2c55-43e9-8907-268c67645358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:08.260055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-5db0582f-2c55-43e9-8907-268c67645358 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:08.261293 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5db0582f-2c55-43e9-8907-268c67645358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:08.288690 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:08.360905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 07:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:08.511424 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4346738-357f-4774-b8fa-18975fa225ef None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:08.512912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d4346738-357f-4774-b8fa-18975fa225ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:08.655117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4346738-357f-4774-b8fa-18975fa225ef None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:08.697043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:08.770140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35af7bf-aee6-4c81-a0fd-30e4188798a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:08.770642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35af7bf-aee6-4c81-a0fd-30e4188798a8 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:08.770916 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35af7bf-aee6-4c81-a0fd-30e4188798a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:08.771184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35af7bf-aee6-4c81-a0fd-30e4188798a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:08.772800 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c35af7bf-aee6-4c81-a0fd-30e4188798a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:08.861164 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35af7bf-aee6-4c81-a0fd-30e4188798a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:08.862748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35af7bf-aee6-4c81-a0fd-30e4188798a8 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:08.880925 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 712/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:10.014207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16935e19-72e1-4593-8a9f-e7b841a001f0 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:10.014770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16935e19-72e1-4593-8a9f-e7b841a001f0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:10.014770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16935e19-72e1-4593-8a9f-e7b841a001f0 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:10.016859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 07:49:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:10.022994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96741721-cfe3-4a5e-ac8b-1a2003b2e73a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:10.023437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96741721-cfe3-4a5e-ac8b-1a2003b2e73a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:10.023602 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96741721-cfe3-4a5e-ac8b-1a2003b2e73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:10.080416 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:10.103166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-96741721-cfe3-4a5e-ac8b-1a2003b2e73a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:10.104252 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-96741721-cfe3-4a5e-ac8b-1a2003b2e73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:10.202132 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 07:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:10.354834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16935e19-72e1-4593-8a9f-e7b841a001f0 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:10.356301 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-16935e19-72e1-4593-8a9f-e7b841a001f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:10.508358 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16935e19-72e1-4593-8a9f-e7b841a001f0 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:10.520408 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:10.621679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d13a4-e7e8-4f25-a12d-cce9f6f23a95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:10.622198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d13a4-e7e8-4f25-a12d-cce9f6f23a95 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:10.622476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d13a4-e7e8-4f25-a12d-cce9f6f23a95 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:10.622748 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d13a4-e7e8-4f25-a12d-cce9f6f23a95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:10.624364 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5e5d13a4-e7e8-4f25-a12d-cce9f6f23a95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:10.713678 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d13a4-e7e8-4f25-a12d-cce9f6f23a95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:10.715291 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d13a4-e7e8-4f25-a12d-cce9f6f23a95 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:10.733795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 709/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:11.867915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ba2ef-b22d-4b8c-abd4-1667ac7137d5 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:11.868484 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ba2ef-b22d-4b8c-abd4-1667ac7137d5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:11.868484 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ba2ef-b22d-4b8c-abd4-1667ac7137d5 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:11.869914 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 07:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:11.877143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee946a1-ae33-49b4-abbe-b7b3e38946b3 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:11.877581 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee946a1-ae33-49b4-abbe-b7b3e38946b3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:11.877673 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee946a1-ae33-49b4-abbe-b7b3e38946b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:11.897875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:11.957086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-cee946a1-ae33-49b4-abbe-b7b3e38946b3 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:11.958308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cee946a1-ae33-49b4-abbe-b7b3e38946b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:12.053828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 07:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:12.202343 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ba2ef-b22d-4b8c-abd4-1667ac7137d5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:12.204622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a38ba2ef-b22d-4b8c-abd4-1667ac7137d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:12.311892 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:12.348411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ba2ef-b22d-4b8c-abd4-1667ac7137d5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:12.478439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9be596-ff50-4b30-be86-19039614506c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:12.479038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9be596-ff50-4b30-be86-19039614506c service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:12.479315 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9be596-ff50-4b30-be86-19039614506c service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:12.479585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9be596-ff50-4b30-be86-19039614506c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:12.481235 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8f9be596-ff50-4b30-be86-19039614506c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:12.557407 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:12.572450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9be596-ff50-4b30-be86-19039614506c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:12.574042 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9be596-ff50-4b30-be86-19039614506c service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:12.591965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 715/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:13.695428 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16821470-515e-48e5-a8c6-625d5a31d97b None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:13.695992 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16821470-515e-48e5-a8c6-625d5a31d97b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:13.695992 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16821470-515e-48e5-a8c6-625d5a31d97b None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:13.697260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 07:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:13.703340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106344e8-7660-4388-b3c8-31f632cdf996 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:13.703884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106344e8-7660-4388-b3c8-31f632cdf996 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:13.703884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106344e8-7660-4388-b3c8-31f632cdf996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:13.778064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-106344e8-7660-4388-b3c8-31f632cdf996 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:13.779263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-106344e8-7660-4388-b3c8-31f632cdf996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:13.874240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 07:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:14.024029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16821470-515e-48e5-a8c6-625d5a31d97b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:14.025605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-16821470-515e-48e5-a8c6-625d5a31d97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:14.110024 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:14.178641 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16821470-515e-48e5-a8c6-625d5a31d97b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:14.292480 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f52a50-54b1-4b5b-bd05-203886002abf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:14.292999 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f52a50-54b1-4b5b-bd05-203886002abf service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:14.293278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f52a50-54b1-4b5b-bd05-203886002abf service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:14.293548 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f52a50-54b1-4b5b-bd05-203886002abf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:14.295142 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-15f52a50-54b1-4b5b-bd05-203886002abf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:14.344053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:14.386570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f52a50-54b1-4b5b-bd05-203886002abf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:14.388208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f52a50-54b1-4b5b-bd05-203886002abf service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:14.407164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 712/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:15.464424 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adfce1-48b4-4432-87bb-81e13bc6499c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:15.464955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adfce1-48b4-4432-87bb-81e13bc6499c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:15.464955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adfce1-48b4-4432-87bb-81e13bc6499c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:15.466319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 07:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:15.472695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8b1d-3111-4b7d-82a7-7244f58b3c93 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:15.473155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8b1d-3111-4b7d-82a7-7244f58b3c93 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:15.473263 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8b1d-3111-4b7d-82a7-7244f58b3c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:15.549421 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-220f8b1d-3111-4b7d-82a7-7244f58b3c93 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:15.550603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-220f8b1d-3111-4b7d-82a7-7244f58b3c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:15.646061 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 07:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:15.795795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adfce1-48b4-4432-87bb-81e13bc6499c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:15.797908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d5adfce1-48b4-4432-87bb-81e13bc6499c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:15.849035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:15.945181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adfce1-48b4-4432-87bb-81e13bc6499c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:16.056486 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96c1ea-11d6-471a-bfda-c990ea8d785c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:16.057087 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96c1ea-11d6-471a-bfda-c990ea8d785c service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:16.057364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96c1ea-11d6-471a-bfda-c990ea8d785c service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:16.057637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96c1ea-11d6-471a-bfda-c990ea8d785c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:16.059233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9b96c1ea-11d6-471a-bfda-c990ea8d785c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:16.066424 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:16.148201 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96c1ea-11d6-471a-bfda-c990ea8d785c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:16.149776 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96c1ea-11d6-471a-bfda-c990ea8d785c service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:16.168490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 718/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:17.241366 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bdb3c-b9f7-4a3e-a167-bf6b7ec41abf None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:17.241939 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bdb3c-b9f7-4a3e-a167-bf6b7ec41abf None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:17.241939 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bdb3c-b9f7-4a3e-a167-bf6b7ec41abf None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:17.243287 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 07:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:17.249319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbfd26-e9f5-4717-b181-48911b7940f9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:17.249722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbfd26-e9f5-4717-b181-48911b7940f9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:17.249865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbfd26-e9f5-4717-b181-48911b7940f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:17.324944 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-24bbfd26-e9f5-4717-b181-48911b7940f9 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:17.326160 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-24bbfd26-e9f5-4717-b181-48911b7940f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:17.394065 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:17.423171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 07:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:17.572898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bdb3c-b9f7-4a3e-a167-bf6b7ec41abf None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:17.574439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e76bdb3c-b9f7-4a3e-a167-bf6b7ec41abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:17.592697 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:17.720558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bdb3c-b9f7-4a3e-a167-bf6b7ec41abf None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:17.808206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:17.834859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa103b3c-c7a9-41d8-8806-247f6b110942 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:17.835466 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa103b3c-c7a9-41d8-8806-247f6b110942 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:17.835766 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa103b3c-c7a9-41d8-8806-247f6b110942 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:17.836034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa103b3c-c7a9-41d8-8806-247f6b110942 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:17.837632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fa103b3c-c7a9-41d8-8806-247f6b110942 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:17.927629 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa103b3c-c7a9-41d8-8806-247f6b110942 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:17.929257 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa103b3c-c7a9-41d8-8806-247f6b110942 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:17.948069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 715/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:18.997991 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a986edc-e3a0-4d2c-bc3f-45c03e72e9bb None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:18.998568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a986edc-e3a0-4d2c-bc3f-45c03e72e9bb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:18.998568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a986edc-e3a0-4d2c-bc3f-45c03e72e9bb None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:18.999902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 07:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:19.006570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b27806-7175-44d2-896b-d2cdaa565ea8 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:19.006999 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b27806-7175-44d2-896b-d2cdaa565ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:19.007101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b27806-7175-44d2-896b-d2cdaa565ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:19.087087 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-47b27806-7175-44d2-896b-d2cdaa565ea8 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:19.088416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-47b27806-7175-44d2-896b-d2cdaa565ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:19.130908 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:19.185990 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 07:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:19.335083 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a986edc-e3a0-4d2c-bc3f-45c03e72e9bb None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:19.336576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8a986edc-e3a0-4d2c-bc3f-45c03e72e9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:19.478726 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a986edc-e3a0-4d2c-bc3f-45c03e72e9bb None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:19.532912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:19.593327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfb7ff-15b0-499f-bdb9-4e36854f379a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:19.593859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfb7ff-15b0-499f-bdb9-4e36854f379a service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:19.594946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfb7ff-15b0-499f-bdb9-4e36854f379a service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:19.595224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfb7ff-15b0-499f-bdb9-4e36854f379a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:19.596864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9adfb7ff-15b0-499f-bdb9-4e36854f379a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:19.684132 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfb7ff-15b0-499f-bdb9-4e36854f379a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:19.685724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfb7ff-15b0-499f-bdb9-4e36854f379a service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:19.703923 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 721/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:20.787930 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45108949-5f23-481a-9107-5f28e761cebe None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:20.788493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45108949-5f23-481a-9107-5f28e761cebe None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:20.788493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45108949-5f23-481a-9107-5f28e761cebe None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:20.789800 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 07:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:20.795937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318d09f-2111-4444-9151-f0de621b80f5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:20.796353 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318d09f-2111-4444-9151-f0de621b80f5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:20.796527 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318d09f-2111-4444-9151-f0de621b80f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:20.874253 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3318d09f-2111-4444-9151-f0de621b80f5 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:20.876349 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3318d09f-2111-4444-9151-f0de621b80f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:20.878546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:20.974368 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 07:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:21.124619 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45108949-5f23-481a-9107-5f28e761cebe None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:21.126213 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-45108949-5f23-481a-9107-5f28e761cebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:21.269703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45108949-5f23-481a-9107-5f28e761cebe None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:21.296255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:21.381678 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c9b19-f889-41fb-a484-c3a4034018f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:21.382281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c9b19-f889-41fb-a484-c3a4034018f3 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:21.382554 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c9b19-f889-41fb-a484-c3a4034018f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:21.382822 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c9b19-f889-41fb-a484-c3a4034018f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:21.384432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cc5c9b19-f889-41fb-a484-c3a4034018f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:21.473978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c9b19-f889-41fb-a484-c3a4034018f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:21.475557 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c9b19-f889-41fb-a484-c3a4034018f3 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:21.492991 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 718/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:23.555528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629128c9-7394-41bf-9492-b3a35914df32 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:23.556107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629128c9-7394-41bf-9492-b3a35914df32 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:23.556107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629128c9-7394-41bf-9492-b3a35914df32 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:23.557426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 07:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:23.562701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b736f7-9227-47e2-b2cc-b42f388dc409 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:23.563154 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b736f7-9227-47e2-b2cc-b42f388dc409 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:23.563251 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b736f7-9227-47e2-b2cc-b42f388dc409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:23.604587 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:23.630095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-00b736f7-9227-47e2-b2cc-b42f388dc409 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:23.631283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-00b736f7-9227-47e2-b2cc-b42f388dc409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:23.737578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 07:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:24.029176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629128c9-7394-41bf-9492-b3a35914df32 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:24.030708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-629128c9-7394-41bf-9492-b3a35914df32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:24.133658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629128c9-7394-41bf-9492-b3a35914df32 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:24.164896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:24.205878 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf48fa-1e99-4ee1-99d8-9152c0d607b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:24.206467 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf48fa-1e99-4ee1-99d8-9152c0d607b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:24.206741 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf48fa-1e99-4ee1-99d8-9152c0d607b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:24.207005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf48fa-1e99-4ee1-99d8-9152c0d607b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:24.208630 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-21bf48fa-1e99-4ee1-99d8-9152c0d607b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:24.315355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf48fa-1e99-4ee1-99d8-9152c0d607b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:24.316992 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf48fa-1e99-4ee1-99d8-9152c0d607b8 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:24.334786 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 724/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:26.648188 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab5749-bf2e-4da8-8461-02b0bb4f4735 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:26.648729 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab5749-bf2e-4da8-8461-02b0bb4f4735 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:26.648729 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab5749-bf2e-4da8-8461-02b0bb4f4735 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:26.650029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 07:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:26.655201 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665d2ae-8383-4ebf-9d54-4ae4f8dcf00a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:26.655585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665d2ae-8383-4ebf-9d54-4ae4f8dcf00a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:26.655752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665d2ae-8383-4ebf-9d54-4ae4f8dcf00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:26.720642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e665d2ae-8383-4ebf-9d54-4ae4f8dcf00a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:26.721845 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e665d2ae-8383-4ebf-9d54-4ae4f8dcf00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:26.734191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:26.818130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 07:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:27.101710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab5749-bf2e-4da8-8461-02b0bb4f4735 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:27.103325 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-63ab5749-bf2e-4da8-8461-02b0bb4f4735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:27.206988 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab5749-bf2e-4da8-8461-02b0bb4f4735 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:27.218904 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:27.279455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cef60b-325b-44f4-a4a4-a54e53b850bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:27.280052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cef60b-325b-44f4-a4a4-a54e53b850bc service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:27.280361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cef60b-325b-44f4-a4a4-a54e53b850bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:27.280631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cef60b-325b-44f4-a4a4-a54e53b850bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:27.282470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d2cef60b-325b-44f4-a4a4-a54e53b850bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:27.391903 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cef60b-325b-44f4-a4a4-a54e53b850bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:27.393455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cef60b-325b-44f4-a4a4-a54e53b850bc service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:27.412200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 721/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:29.886880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18cf0b-31e1-4c34-9346-8565b1e9c054 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:29.887429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18cf0b-31e1-4c34-9346-8565b1e9c054 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:29.887429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18cf0b-31e1-4c34-9346-8565b1e9c054 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:29.888810 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 07:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:29.894574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbf848-ca6b-4348-8367-f38d46d9969b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:29.895018 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbf848-ca6b-4348-8367-f38d46d9969b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:29.895125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbf848-ca6b-4348-8367-f38d46d9969b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:29.943381 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:29.961502 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c3bbf848-ca6b-4348-8367-f38d46d9969b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:29.962712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c3bbf848-ca6b-4348-8367-f38d46d9969b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:30.058333 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 07:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:30.344804 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18cf0b-31e1-4c34-9346-8565b1e9c054 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:30.346340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4f18cf0b-31e1-4c34-9346-8565b1e9c054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:30.450529 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18cf0b-31e1-4c34-9346-8565b1e9c054 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:30.489811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:30.523051 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa3a8c-5fdd-494e-9461-203a63fefe93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:30.523643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa3a8c-5fdd-494e-9461-203a63fefe93 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:30.523916 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa3a8c-5fdd-494e-9461-203a63fefe93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:30.524210 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa3a8c-5fdd-494e-9461-203a63fefe93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:30.525787 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-35aa3a8c-5fdd-494e-9461-203a63fefe93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:30.630655 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa3a8c-5fdd-494e-9461-203a63fefe93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:30.632067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa3a8c-5fdd-494e-9461-203a63fefe93 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:30.649917 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 727/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:32.941316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861ec53-e814-4b66-a6d3-69212946ab11 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:32.941875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861ec53-e814-4b66-a6d3-69212946ab11 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:32.941875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861ec53-e814-4b66-a6d3-69212946ab11 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:32.943275 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 07:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:32.948707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00581988-3191-44c2-85a7-a2c2ef77f3e1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:32.949122 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00581988-3191-44c2-85a7-a2c2ef77f3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:32.949252 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00581988-3191-44c2-85a7-a2c2ef77f3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:33.016177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-00581988-3191-44c2-85a7-a2c2ef77f3e1 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:33.017384 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-00581988-3191-44c2-85a7-a2c2ef77f3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:33.037721 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:33.113234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 07:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:33.397847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861ec53-e814-4b66-a6d3-69212946ab11 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:33.399388 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7861ec53-e814-4b66-a6d3-69212946ab11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:33.508717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861ec53-e814-4b66-a6d3-69212946ab11 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:33.529274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:33.583960 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34433c-8fd2-4d6b-8734-62ae37765213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:33.584584 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34433c-8fd2-4d6b-8734-62ae37765213 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:33.584863 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34433c-8fd2-4d6b-8734-62ae37765213 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:33.585128 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34433c-8fd2-4d6b-8734-62ae37765213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:33.586695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fb34433c-8fd2-4d6b-8734-62ae37765213 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:33.695424 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb34433c-8fd2-4d6b-8734-62ae37765213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:33.696805 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb34433c-8fd2-4d6b-8734-62ae37765213 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:33.714906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 724/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:35.925620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c06e9-b942-47ec-8557-3de38b150cbd None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:35.926190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c06e9-b942-47ec-8557-3de38b150cbd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:35.926190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c06e9-b942-47ec-8557-3de38b150cbd None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:35.927517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 07:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:35.933080 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f90180-be43-4533-97ef-cc44e72e86e2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:35.933474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f90180-be43-4533-97ef-cc44e72e86e2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:35.933638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f90180-be43-4533-97ef-cc44e72e86e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:35.998812 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:36.002585 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e6f90180-be43-4533-97ef-cc44e72e86e2 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:36.004029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e6f90180-be43-4533-97ef-cc44e72e86e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:36.099214 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 07:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:36.381232 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c06e9-b942-47ec-8557-3de38b150cbd None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:36.382905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0c1c06e9-b942-47ec-8557-3de38b150cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:36.486003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c06e9-b942-47ec-8557-3de38b150cbd None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:36.533561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:36.557702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c71ec-318e-4452-a8ff-7825aae97a60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:36.558300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c71ec-318e-4452-a8ff-7825aae97a60 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:36.558577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c71ec-318e-4452-a8ff-7825aae97a60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:36.558847 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c71ec-318e-4452-a8ff-7825aae97a60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:36.560469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a59c71ec-318e-4452-a8ff-7825aae97a60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:36.666769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c71ec-318e-4452-a8ff-7825aae97a60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:36.668215 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c71ec-318e-4452-a8ff-7825aae97a60 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:36.685850 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 730/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 07:49:38.923625 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68806656-45b4-4c75-8029-8ba1d9f61c3e None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:38.924183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68806656-45b4-4c75-8029-8ba1d9f61c3e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:38.924183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68806656-45b4-4c75-8029-8ba1d9f61c3e None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:38.925551 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 07:49:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:38.930840 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bccf27-04f2-4360-91ee-b7cd74862138 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:38.931232 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bccf27-04f2-4360-91ee-b7cd74862138 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:38.931386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bccf27-04f2-4360-91ee-b7cd74862138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:38.997594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-18bccf27-04f2-4360-91ee-b7cd74862138 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:38.998785 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-18bccf27-04f2-4360-91ee-b7cd74862138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:39.035448 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:39.118641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 07:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:39.404713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68806656-45b4-4c75-8029-8ba1d9f61c3e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:39.406835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-68806656-45b4-4c75-8029-8ba1d9f61c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:39.513336 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68806656-45b4-4c75-8029-8ba1d9f61c3e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:39.544296 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:39.590603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375a7c5-1ba3-43e7-98bc-5f1e63e4135f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:39.591311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375a7c5-1ba3-43e7-98bc-5f1e63e4135f service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:39.591591 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375a7c5-1ba3-43e7-98bc-5f1e63e4135f service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:39.591857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375a7c5-1ba3-43e7-98bc-5f1e63e4135f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:39.593480 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a375a7c5-1ba3-43e7-98bc-5f1e63e4135f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:39.699741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375a7c5-1ba3-43e7-98bc-5f1e63e4135f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:39.701124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375a7c5-1ba3-43e7-98bc-5f1e63e4135f service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:39.719013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 727/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:42.092166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abbf0b-c30c-415d-928d-c6fcf7b2045c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:42.092725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abbf0b-c30c-415d-928d-c6fcf7b2045c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:42.092725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abbf0b-c30c-415d-928d-c6fcf7b2045c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:42.095379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 07:49:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:42.100831 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0984c-34da-408f-8cb1-f93895edf899 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:42.101272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0984c-34da-408f-8cb1-f93895edf899 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:42.101519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0984c-34da-408f-8cb1-f93895edf899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:42.173206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-56f0984c-34da-408f-8cb1-f93895edf899 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:42.174406 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-56f0984c-34da-408f-8cb1-f93895edf899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:42.182626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:42.269879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 07:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:42.553872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abbf0b-c30c-415d-928d-c6fcf7b2045c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:42.555460 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-94abbf0b-c30c-415d-928d-c6fcf7b2045c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:42.657796 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abbf0b-c30c-415d-928d-c6fcf7b2045c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:42.711672 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:42.729713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c1a51-4630-4052-99da-5e68812dbf43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:42.730310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c1a51-4630-4052-99da-5e68812dbf43 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:42.730587 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c1a51-4630-4052-99da-5e68812dbf43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:42.730863 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c1a51-4630-4052-99da-5e68812dbf43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:42.732485 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4b9c1a51-4630-4052-99da-5e68812dbf43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:42.842857 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c1a51-4630-4052-99da-5e68812dbf43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:42.845066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c1a51-4630-4052-99da-5e68812dbf43 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:42.864805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 733/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:45.073579 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e332176-f148-4448-9f62-15fc9bdde484 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:45.074202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e332176-f148-4448-9f62-15fc9bdde484 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:45.074202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e332176-f148-4448-9f62-15fc9bdde484 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:45.075516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 07:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:45.081504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718cecd-e026-4fe4-810c-3fa931cca5c9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:45.081971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718cecd-e026-4fe4-810c-3fa931cca5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:45.082083 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718cecd-e026-4fe4-810c-3fa931cca5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:45.148318 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a718cecd-e026-4fe4-810c-3fa931cca5c9 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:45.149541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a718cecd-e026-4fe4-810c-3fa931cca5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:45.187821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:45.246262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 07:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:45.529033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e332176-f148-4448-9f62-15fc9bdde484 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:45.530600 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1e332176-f148-4448-9f62-15fc9bdde484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:45.635210 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e332176-f148-4448-9f62-15fc9bdde484 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:45.672914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:45.709354 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e55b83-43a0-4a0e-ba6a-689ed76996af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:45.709948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e55b83-43a0-4a0e-ba6a-689ed76996af service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:45.710222 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e55b83-43a0-4a0e-ba6a-689ed76996af service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:45.710489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e55b83-43a0-4a0e-ba6a-689ed76996af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:45.712055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-65e55b83-43a0-4a0e-ba6a-689ed76996af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:45.825334 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e55b83-43a0-4a0e-ba6a-689ed76996af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:45.826774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e55b83-43a0-4a0e-ba6a-689ed76996af service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:45.845155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 730/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:48.023009 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b711a6-081a-42be-aef9-fe64af0fbde0 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:48.023609 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b711a6-081a-42be-aef9-fe64af0fbde0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:48.023609 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b711a6-081a-42be-aef9-fe64af0fbde0 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:48.024910 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 07:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:48.030113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed02f1f-f48a-4e36-885d-2e6c86a9b888 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:48.030534 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed02f1f-f48a-4e36-885d-2e6c86a9b888 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:48.030668 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed02f1f-f48a-4e36-885d-2e6c86a9b888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:48.096617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-5ed02f1f-f48a-4e36-885d-2e6c86a9b888 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:48.097820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5ed02f1f-f48a-4e36-885d-2e6c86a9b888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:48.115980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:48.194287 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 07:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:48.481511 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b711a6-081a-42be-aef9-fe64af0fbde0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:48.483077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-45b711a6-081a-42be-aef9-fe64af0fbde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:48.701514 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b711a6-081a-42be-aef9-fe64af0fbde0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:48.766192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:48.772947 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166eb493-665b-428c-8447-540f1c414f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:48.773542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166eb493-665b-428c-8447-540f1c414f45 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:48.773813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166eb493-665b-428c-8447-540f1c414f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:48.774078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166eb493-665b-428c-8447-540f1c414f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:48.775658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-166eb493-665b-428c-8447-540f1c414f45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:48.883396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166eb493-665b-428c-8447-540f1c414f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:48.884852 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166eb493-665b-428c-8447-540f1c414f45 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:48.902410 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 736/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:49:51.107157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ad45b-ee0c-4c20-8c2d-f948b44ec77a None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:51.107728 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ad45b-ee0c-4c20-8c2d-f948b44ec77a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:51.107728 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ad45b-ee0c-4c20-8c2d-f948b44ec77a None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:51.109128 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 07:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:49:51.115274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fc7f6-5281-48aa-9fd0-19eb99d2d8ff None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:51.115679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fc7f6-5281-48aa-9fd0-19eb99d2d8ff None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:51.115818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fc7f6-5281-48aa-9fd0-19eb99d2d8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:51.180803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-fd1fc7f6-5281-48aa-9fd0-19eb99d2d8ff None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:49:51.182000 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fd1fc7f6-5281-48aa-9fd0-19eb99d2d8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:51.229129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:51.276700 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 07:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:49:51.559435 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ad45b-ee0c-4c20-8c2d-f948b44ec77a None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:49:51.560996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9a3ad45b-ee0c-4c20-8c2d-f948b44ec77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:51.663595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ad45b-ee0c-4c20-8c2d-f948b44ec77a None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:49:51.710781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:49:51.737526 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63db05-7914-4199-8fb0-402c9c21d164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:49:51.738120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63db05-7914-4199-8fb0-402c9c21d164 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:49:51.738391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63db05-7914-4199-8fb0-402c9c21d164 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:49:51.738655 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63db05-7914-4199-8fb0-402c9c21d164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:49:51.740248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1b63db05-7914-4199-8fb0-402c9c21d164 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:49:51.848699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63db05-7914-4199-8fb0-402c9c21d164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:49:51.850176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63db05-7914-4199-8fb0-402c9c21d164 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:49:51.868446 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 733/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:49:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:50:01.274964 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=22 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:50:01.866240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=66 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:50:12.429144 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fcee0-a684-475b-bba0-93a9eb727fb3 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:12.429664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fcee0-a684-475b-bba0-93a9eb727fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:12.429768 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fcee0-a684-475b-bba0-93a9eb727fb3 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:12.431041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 738/1471] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:50:12.436559 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150411a6-739e-4f99-b6ad-d504926ccc65 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:12.437546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150411a6-739e-4f99-b6ad-d504926ccc65 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:12.437726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150411a6-739e-4f99-b6ad-d504926ccc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:12.504301 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-150411a6-739e-4f99-b6ad-d504926ccc65 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:50:12.505421 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-150411a6-739e-4f99-b6ad-d504926ccc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:12.598942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 734/1472] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:50:12.899323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd267d5a-7a43-4665-887b-68ee162d63ce None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:12.899607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd267d5a-7a43-4665-887b-68ee162d63ce None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:12.899880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd267d5a-7a43-4665-887b-68ee162d63ce None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:12.901228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 739/1473] 10.222.0.26 () {54 vars in 961 bytes} [Wed Sep 7 07:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:50:12.906335 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a598411-0ee8-4275-acce-f1260b604907 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:12.906615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a598411-0ee8-4275-acce-f1260b604907 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:12.906884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a598411-0ee8-4275-acce-f1260b604907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:12.985072 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-6a598411-0ee8-4275-acce-f1260b604907 None None] MFA Rules not processed for user `9b0652c89c9945e2866512c9da4793ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:50:12.986308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6a598411-0ee8-4275-acce-f1260b604907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:13.006311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:50:13.061503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 735/1474] 10.222.0.26 () {58 vars in 1058 bytes} [Wed Sep 7 07:50:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:50:13.069605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd267d5a-7a43-4665-887b-68ee162d63ce None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:50:13.071103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-dd267d5a-7a43-4665-887b-68ee162d63ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:13.174011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd267d5a-7a43-4665-887b-68ee162d63ce None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:50:13.244624 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c214576-af2f-40ec-b9b2-d9d50176c748 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b0652c89c9945e2866512c9da4793ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:50:13.245207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c214576-af2f-40ec-b9b2-d9d50176c748 service glance] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:13.245477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c214576-af2f-40ec-b9b2-d9d50176c748 service glance] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:13.245741 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c214576-af2f-40ec-b9b2-d9d50176c748 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:13.247314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8c214576-af2f-40ec-b9b2-d9d50176c748 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:13.299240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:50:13.353017 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c214576-af2f-40ec-b9b2-d9d50176c748 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:50:13.354359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c214576-af2f-40ec-b9b2-d9d50176c748 service glance] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:50:13.372779 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 740/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:50:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:50:15.681423 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c35de-8d42-49ed-80d5-36baefe09e35 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:15.681998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c35de-8d42-49ed-80d5-36baefe09e35 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:15.681998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c35de-8d42-49ed-80d5-36baefe09e35 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:15.683254 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 736/1476] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:50:15.689297 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516698a-63a0-4bc4-9a8c-95523a684016 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:15.689765 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516698a-63a0-4bc4-9a8c-95523a684016 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:15.689873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516698a-63a0-4bc4-9a8c-95523a684016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:15.759384 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7516698a-63a0-4bc4-9a8c-95523a684016 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:50:15.760678 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7516698a-63a0-4bc4-9a8c-95523a684016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:15.859552 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 741/1477] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:50:16.134889 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c35de-8d42-49ed-80d5-36baefe09e35 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:50:16.136469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e56c35de-8d42-49ed-80d5-36baefe09e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:16.261641 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c35de-8d42-49ed-80d5-36baefe09e35 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:50:16.315068 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:50:16.360499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2db5a-6873-4293-ab05-ba2e5a661e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:50:16.361097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2db5a-6873-4293-ab05-ba2e5a661e07 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:16.361371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2db5a-6873-4293-ab05-ba2e5a661e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:16.361634 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2db5a-6873-4293-ab05-ba2e5a661e07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:16.362258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c2db5a-6873-4293-ab05-ba2e5a661e07 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:50:16.364040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: WARNING py.warnings [None req-53c2db5a-6873-4293-ab05-ba2e5a661e07 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 07:50:16.364040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: warnings.warn(msg) Sep 07 07:50:16.364040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: Sep 07 07:50:16.365328 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c2db5a-6873-4293-ab05-ba2e5a661e07 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:50:16.384996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 737/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 7 07:50:16 2022] GET /identity/v3/projects => generated 2553 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:50:18.562561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a604aad-6dc3-490f-b50c-2cd89316af38 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:18.563118 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a604aad-6dc3-490f-b50c-2cd89316af38 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:18.563118 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a604aad-6dc3-490f-b50c-2cd89316af38 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:18.564511 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 742/1479] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:50:18.569946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184e233-9d74-49e8-8320-7f0d16dd1cbc None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:18.570342 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184e233-9d74-49e8-8320-7f0d16dd1cbc None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:18.570503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184e233-9d74-49e8-8320-7f0d16dd1cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:18.642414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1184e233-9d74-49e8-8320-7f0d16dd1cbc None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:50:18.643559 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1184e233-9d74-49e8-8320-7f0d16dd1cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:18.740804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 738/1480] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:50:19.041279 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a604aad-6dc3-490f-b50c-2cd89316af38 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:50:19.043113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0a604aad-6dc3-490f-b50c-2cd89316af38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:19.053185 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:50:19.195268 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a604aad-6dc3-490f-b50c-2cd89316af38 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:50:19.274002 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:50:19.307201 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35990e1-b058-402a-a04f-aaf48bea163d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:50:19.307719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35990e1-b058-402a-a04f-aaf48bea163d service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:19.307991 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35990e1-b058-402a-a04f-aaf48bea163d service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:19.308349 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35990e1-b058-402a-a04f-aaf48bea163d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:19.309937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c35990e1-b058-402a-a04f-aaf48bea163d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:19.397480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35990e1-b058-402a-a04f-aaf48bea163d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:50:19.399051 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35990e1-b058-402a-a04f-aaf48bea163d service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:50:19.417037 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 743/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:50:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:50:21.758495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50218de-d72a-4297-a58d-597e5e14f559 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:21.759042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50218de-d72a-4297-a58d-597e5e14f559 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:21.759042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50218de-d72a-4297-a58d-597e5e14f559 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:21.760356 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 739/1482] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:50:21.766582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221153a-ecc6-4c6d-9e0b-918236a754ba None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:21.767023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221153a-ecc6-4c6d-9e0b-918236a754ba None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:21.767122 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221153a-ecc6-4c6d-9e0b-918236a754ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:21.833042 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0221153a-ecc6-4c6d-9e0b-918236a754ba None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:50:21.834263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0221153a-ecc6-4c6d-9e0b-918236a754ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:21.881870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:50:21.932730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 744/1483] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:50:22.228328 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50218de-d72a-4297-a58d-597e5e14f559 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:50:22.229872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c50218de-d72a-4297-a58d-597e5e14f559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:22.253581 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:50:22.333383 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50218de-d72a-4297-a58d-597e5e14f559 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:50:22.406428 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b863b1-0ffd-4b0d-9fcd-ff599365da7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:50:22.407031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b863b1-0ffd-4b0d-9fcd-ff599365da7e service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:22.407306 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b863b1-0ffd-4b0d-9fcd-ff599365da7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:22.407574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b863b1-0ffd-4b0d-9fcd-ff599365da7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:22.409185 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f2b863b1-0ffd-4b0d-9fcd-ff599365da7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:22.484722 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:50:22.520015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b863b1-0ffd-4b0d-9fcd-ff599365da7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:50:22.521469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b863b1-0ffd-4b0d-9fcd-ff599365da7e service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:50:22.541280 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 740/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:50:24.739467 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb84c77-89ec-4c08-bb05-68264c763fd3 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:24.740113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb84c77-89ec-4c08-bb05-68264c763fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:24.740113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb84c77-89ec-4c08-bb05-68264c763fd3 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:24.741295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 745/1485] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:50:24.746612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf89a3f-639e-4b1c-b039-0556e8971d16 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:24.747039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf89a3f-639e-4b1c-b039-0556e8971d16 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:24.747160 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf89a3f-639e-4b1c-b039-0556e8971d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:24.813121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-faf89a3f-639e-4b1c-b039-0556e8971d16 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:50:24.814324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-faf89a3f-639e-4b1c-b039-0556e8971d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:24.909525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 741/1486] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:50:25.208263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb84c77-89ec-4c08-bb05-68264c763fd3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:50:25.209867 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1eb84c77-89ec-4c08-bb05-68264c763fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:25.311153 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb84c77-89ec-4c08-bb05-68264c763fd3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:50:25.395803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:50:25.397834 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e03466-0e00-4bae-8df0-ab92719546cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:50:25.398492 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e03466-0e00-4bae-8df0-ab92719546cd service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:25.398991 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e03466-0e00-4bae-8df0-ab92719546cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:25.399450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e03466-0e00-4bae-8df0-ab92719546cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:25.401443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-30e03466-0e00-4bae-8df0-ab92719546cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:25.511896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e03466-0e00-4bae-8df0-ab92719546cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:50:25.513268 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e03466-0e00-4bae-8df0-ab92719546cd service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:50:25.532230 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 746/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:50:27.877880 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa1e0f-ba55-4db2-9e41-2592c9bd8dca None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:27.878457 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa1e0f-ba55-4db2-9e41-2592c9bd8dca None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:27.878457 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa1e0f-ba55-4db2-9e41-2592c9bd8dca None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:27.879687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 742/1488] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:50:27.885932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553abfa-1dc3-438d-96c9-d8ea92f7f0b6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:27.886378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553abfa-1dc3-438d-96c9-d8ea92f7f0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:27.886477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553abfa-1dc3-438d-96c9-d8ea92f7f0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:27.952855 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9553abfa-1dc3-438d-96c9-d8ea92f7f0b6 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:50:27.954070 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9553abfa-1dc3-438d-96c9-d8ea92f7f0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:28.007272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:50:28.051126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 747/1489] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:50:28.341761 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa1e0f-ba55-4db2-9e41-2592c9bd8dca None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:50:28.343843 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9afa1e0f-ba55-4db2-9e41-2592c9bd8dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:28.392447 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:50:28.460658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa1e0f-ba55-4db2-9e41-2592c9bd8dca None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:50:28.533365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955589df-c9be-4f77-9b7c-e41fbfc62395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:50:28.533964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955589df-c9be-4f77-9b7c-e41fbfc62395 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:28.534244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955589df-c9be-4f77-9b7c-e41fbfc62395 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:28.534518 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955589df-c9be-4f77-9b7c-e41fbfc62395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:28.536170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-955589df-c9be-4f77-9b7c-e41fbfc62395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:28.614866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:50:28.644101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955589df-c9be-4f77-9b7c-e41fbfc62395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:50:28.645563 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955589df-c9be-4f77-9b7c-e41fbfc62395 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:50:28.663414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 743/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:50:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:50:31.063903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978d54f-dfcc-4866-a99c-2e9f9761ed94 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:31.064490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978d54f-dfcc-4866-a99c-2e9f9761ed94 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:31.064490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978d54f-dfcc-4866-a99c-2e9f9761ed94 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:31.065814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 748/1491] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 7 07:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:50:31.071282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e21198-7ea3-4d1f-beda-fe96fd180011 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:31.071683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e21198-7ea3-4d1f-beda-fe96fd180011 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:31.071833 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e21198-7ea3-4d1f-beda-fe96fd180011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:31.149017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-18e21198-7ea3-4d1f-beda-fe96fd180011 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:50:31.150240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-18e21198-7ea3-4d1f-beda-fe96fd180011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:31.362361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 744/1492] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 7 07:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:50:31.653674 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978d54f-dfcc-4866-a99c-2e9f9761ed94 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:50:31.655250 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2978d54f-dfcc-4866-a99c-2e9f9761ed94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:31.758043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978d54f-dfcc-4866-a99c-2e9f9761ed94 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:50:31.829276 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f51716-35aa-4c6f-8098-e4562d2bc193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:50:31.829872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f51716-35aa-4c6f-8098-e4562d2bc193 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:50:31.830148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f51716-35aa-4c6f-8098-e4562d2bc193 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:50:31.830416 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f51716-35aa-4c6f-8098-e4562d2bc193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:50:31.832018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-42f51716-35aa-4c6f-8098-e4562d2bc193 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:50:31.840888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:50:31.942486 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f51716-35aa-4c6f-8098-e4562d2bc193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:50:31.943811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f51716-35aa-4c6f-8098-e4562d2bc193 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:50:31.961943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 749/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:50:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:50:41.360520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=82 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:50:41.959690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=50 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:51:46.229304 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb901c-b062-4290-a4b1-dfaeff58dc2a None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:46.229901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb901c-b062-4290-a4b1-dfaeff58dc2a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:46.229901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb901c-b062-4290-a4b1-dfaeff58dc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:46.304625 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-bddb901c-b062-4290-a4b1-dfaeff58dc2a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:51:46.305885 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-bddb901c-b062-4290-a4b1-dfaeff58dc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:51:46.409035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 745/1494] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:51:46.417099 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a7339-0a4c-4abc-b389-97afa4015ece None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:46.417599 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a7339-0a4c-4abc-b389-97afa4015ece None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:46.417693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a7339-0a4c-4abc-b389-97afa4015ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:46.492419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3d6a7339-0a4c-4abc-b389-97afa4015ece None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:51:46.493583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3d6a7339-0a4c-4abc-b389-97afa4015ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:51:46.590885 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 750/1495] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:51:46.595664 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb901c-b062-4290-a4b1-dfaeff58dc2a None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:51:46.596955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-bddb901c-b062-4290-a4b1-dfaeff58dc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:51:46.608433 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb901c-b062-4290-a4b1-dfaeff58dc2a None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:51:46.670058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:51:46.704255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1287938-5c03-4cdc-95f3-16f14079e31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:51:46.704851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1287938-5c03-4cdc-95f3-16f14079e31a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:46.705128 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1287938-5c03-4cdc-95f3-16f14079e31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:46.705397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1287938-5c03-4cdc-95f3-16f14079e31a admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:46.706021 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1287938-5c03-4cdc-95f3-16f14079e31a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:51:46.706319 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1287938-5c03-4cdc-95f3-16f14079e31a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:51:46.707326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: WARNING py.warnings [None req-f1287938-5c03-4cdc-95f3-16f14079e31a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 07:51:46.707326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: warnings.warn(msg) Sep 07 07:51:46.707326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: Sep 07 07:51:46.708225 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1287938-5c03-4cdc-95f3-16f14079e31a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:51:46.720106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 746/1496] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:51:46.728478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a7339-0a4c-4abc-b389-97afa4015ece None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:51:46.738274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a7339-0a4c-4abc-b389-97afa4015ece None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:51:46.799103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:51:46.833624 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612cb6f-d8f7-4e64-b6d5-03fdef0f6658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:51:46.834222 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612cb6f-d8f7-4e64-b6d5-03fdef0f6658 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:46.834494 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612cb6f-d8f7-4e64-b6d5-03fdef0f6658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:46.834759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612cb6f-d8f7-4e64-b6d5-03fdef0f6658 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:46.835382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612cb6f-d8f7-4e64-b6d5-03fdef0f6658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:51:46.837115 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-8612cb6f-d8f7-4e64-b6d5-03fdef0f6658 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 07:51:46.837115 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Sep 07 07:51:46.837115 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: Sep 07 07:51:46.838286 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612cb6f-d8f7-4e64-b6d5-03fdef0f6658 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:51:46.892623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 751/1497] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:51:46 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:51:46.896743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1287938-5c03-4cdc-95f3-16f14079e31a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:51:46.898304 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f1287938-5c03-4cdc-95f3-16f14079e31a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:51:46.931434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1287938-5c03-4cdc-95f3-16f14079e31a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:51:47.026556 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75148bc1-97e4-4df7-a275-937796fc5b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:51:47.027152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75148bc1-97e4-4df7-a275-937796fc5b5d admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:47.027425 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75148bc1-97e4-4df7-a275-937796fc5b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:47.027692 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75148bc1-97e4-4df7-a275-937796fc5b5d admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:47.028926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75148bc1-97e4-4df7-a275-937796fc5b5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:51:47.029934 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: WARNING py.warnings [None req-75148bc1-97e4-4df7-a275-937796fc5b5d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 07:51:47.029934 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: warnings.warn(msg) Sep 07 07:51:47.029934 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: Sep 07 07:51:47.031118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75148bc1-97e4-4df7-a275-937796fc5b5d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:51:47.072674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 747/1498] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:51:46 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:51:47.077052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612cb6f-d8f7-4e64-b6d5-03fdef0f6658 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:51:47.078599 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8612cb6f-d8f7-4e64-b6d5-03fdef0f6658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:51:47.139916 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612cb6f-d8f7-4e64-b6d5-03fdef0f6658 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:51:47.222951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:51:47.234745 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024f888-5c96-423f-8322-5d0d7b96f38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:51:47.235368 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024f888-5c96-423f-8322-5d0d7b96f38e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:47.235643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024f888-5c96-423f-8322-5d0d7b96f38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:47.235910 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024f888-5c96-423f-8322-5d0d7b96f38e admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:47.236573 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024f888-5c96-423f-8322-5d0d7b96f38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:51:47.237580 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-4024f888-5c96-423f-8322-5d0d7b96f38e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 07:51:47.237580 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Sep 07 07:51:47.237580 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: Sep 07 07:51:47.238458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024f888-5c96-423f-8322-5d0d7b96f38e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:51:47.250960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 752/1499] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:51:47.255081 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75148bc1-97e4-4df7-a275-937796fc5b5d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:51:47.256601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-75148bc1-97e4-4df7-a275-937796fc5b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:51:47.268223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75148bc1-97e4-4df7-a275-937796fc5b5d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:51:47.293740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:51:47.364013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1ad57-5a2a-4971-af79-34687b5a45fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:51:47.364572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1ad57-5a2a-4971-af79-34687b5a45fc admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:47.364844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1ad57-5a2a-4971-af79-34687b5a45fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:47.365148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1ad57-5a2a-4971-af79-34687b5a45fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:47.365774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1ad57-5a2a-4971-af79-34687b5a45fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:51:47.366774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: WARNING py.warnings [None req-08e1ad57-5a2a-4971-af79-34687b5a45fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 07:51:47.366774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: warnings.warn(msg) Sep 07 07:51:47.366774 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: Sep 07 07:51:47.367645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1ad57-5a2a-4971-af79-34687b5a45fc admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:51:47.379587 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 748/1500] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:51:47.383726 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024f888-5c96-423f-8322-5d0d7b96f38e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:51:47.385298 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4024f888-5c96-423f-8322-5d0d7b96f38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:51:47.419535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024f888-5c96-423f-8322-5d0d7b96f38e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:51:47.510431 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7da609-2182-4920-a5d6-a486bb9a5f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:51:47.511067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7da609-2182-4920-a5d6-a486bb9a5f57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:47.511340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7da609-2182-4920-a5d6-a486bb9a5f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:47.511604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7da609-2182-4920-a5d6-a486bb9a5f57 admin admin] PATH_INFO: `/v3/projects/37c588e72d504829b6b7d34e0a823d20/users/da92e084cf1348e8ae5ba5967c7c14fd/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:47.533559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7da609-2182-4920-a5d6-a486bb9a5f57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37c588e72d504829b6b7d34e0a823d20, user_id=da92e084cf1348e8ae5ba5967c7c14fd, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:51:47.534556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-8f7da609-2182-4920-a5d6-a486bb9a5f57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 07:51:47.534556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Sep 07 07:51:47.534556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: Sep 07 07:51:47.536521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7da609-2182-4920-a5d6-a486bb9a5f57 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:51:47.560923 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 753/1501] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:51:47 2022] PUT /identity/v3/projects/37c588e72d504829b6b7d34e0a823d20/users/da92e084cf1348e8ae5ba5967c7c14fd/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:51:47.573321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856324de-8b13-4db6-80fb-41e380de7524 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:47.573608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856324de-8b13-4db6-80fb-41e380de7524 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:47.573885 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856324de-8b13-4db6-80fb-41e380de7524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:47.631839 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-856324de-8b13-4db6-80fb-41e380de7524 None None] MFA Rules not processed for user `da92e084cf1348e8ae5ba5967c7c14fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:51:47.633091 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-856324de-8b13-4db6-80fb-41e380de7524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:51:47.712000 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 749/1502] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:51:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:51:47.726026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e0963-6807-4e87-a8af-293b8e5d6b9f None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:47.726306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e0963-6807-4e87-a8af-293b8e5d6b9f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:47.726574 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e0963-6807-4e87-a8af-293b8e5d6b9f None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:47.727897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 754/1503] 10.222.0.26 () {54 vars in 964 bytes} [Wed Sep 7 07:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:51:47.732720 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfeea85-8b01-4027-868f-aca9bdb48bbd None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:47.733004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfeea85-8b01-4027-868f-aca9bdb48bbd None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:47.733275 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfeea85-8b01-4027-868f-aca9bdb48bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:47.765183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:51:47.808722 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-cdfeea85-8b01-4027-868f-aca9bdb48bbd None None] MFA Rules not processed for user `92699b512c064b05948afe597650fdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:51:47.810460 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cdfeea85-8b01-4027-868f-aca9bdb48bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:51:47.888993 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 750/1504] 10.222.0.26 () {58 vars in 1061 bytes} [Wed Sep 7 07:51:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:51:47.900455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e0963-6807-4e87-a8af-293b8e5d6b9f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:51:47.902204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-903e0963-6807-4e87-a8af-293b8e5d6b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:51:47.952536 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:51:48.005873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e0963-6807-4e87-a8af-293b8e5d6b9f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:51:48.078455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42af09a-bd8f-49b0-8b70-0bf032127a66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:51:48.079041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42af09a-bd8f-49b0-8b70-0bf032127a66 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:48.079314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42af09a-bd8f-49b0-8b70-0bf032127a66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:48.079584 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42af09a-bd8f-49b0-8b70-0bf032127a66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:48.081192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f42af09a-bd8f-49b0-8b70-0bf032127a66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:51:48.178466 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42af09a-bd8f-49b0-8b70-0bf032127a66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:51:48.179791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42af09a-bd8f-49b0-8b70-0bf032127a66 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:51:48.183236 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:51:48.198757 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 755/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:51:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:51:48.243501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9ed41-fe6f-4f04-bcaf-3fa2abf4be3b None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:48.243782 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9ed41-fe6f-4f04-bcaf-3fa2abf4be3b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:48.244072 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9ed41-fe6f-4f04-bcaf-3fa2abf4be3b None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:48.245347 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 751/1506] 10.222.0.26 () {52 vars in 830 bytes} [Wed Sep 7 07:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:51:48.249695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42af09a-bd8f-49b0-8b70-0bf032127a66 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:51:48.250967 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f42af09a-bd8f-49b0-8b70-0bf032127a66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:51:48.348922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42af09a-bd8f-49b0-8b70-0bf032127a66 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:51:48.441266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e681f-2b99-4235-aac9-74c1dedef822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:51:48.441883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e681f-2b99-4235-aac9-74c1dedef822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:48.442077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e681f-2b99-4235-aac9-74c1dedef822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:48.442599 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e681f-2b99-4235-aac9-74c1dedef822 admin admin] PATH_INFO: `/v3/users/da92e084cf1348e8ae5ba5967c7c14fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:48.454519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e681f-2b99-4235-aac9-74c1dedef822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da92e084cf1348e8ae5ba5967c7c14fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:51:48.455528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-de5e681f-2b99-4235-aac9-74c1dedef822 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 07:51:48.455528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Sep 07 07:51:48.455528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: Sep 07 07:51:48.456778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e681f-2b99-4235-aac9-74c1dedef822 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:51:48.466340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:51:48.523063 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-de5e681f-2b99-4235-aac9-74c1dedef822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da92e084cf1348e8ae5ba5967c7c14fd'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:51:48.546962 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-de5e681f-2b99-4235-aac9-74c1dedef822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da92e084cf1348e8ae5ba5967c7c14fd'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:51:48.556000 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-de5e681f-2b99-4235-aac9-74c1dedef822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da92e084cf1348e8ae5ba5967c7c14fd'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:51:48.556609 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-de5e681f-2b99-4235-aac9-74c1dedef822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da92e084cf1348e8ae5ba5967c7c14fd'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:51:48.570994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-de5e681f-2b99-4235-aac9-74c1dedef822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da92e084cf1348e8ae5ba5967c7c14fd'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:51:48.577117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 756/1507] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:51:48 2022] DELETE /identity/v3/users/da92e084cf1348e8ae5ba5967c7c14fd => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:51:48.581446 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8a65e-d8d8-4774-a464-b15306f1a631 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:48.581890 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8a65e-d8d8-4774-a464-b15306f1a631 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:48.581995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8a65e-d8d8-4774-a464-b15306f1a631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:48.639568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9cf8a65e-d8d8-4774-a464-b15306f1a631 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:51:48.640815 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9cf8a65e-d8d8-4774-a464-b15306f1a631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:51:48.717700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:51:48.730459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 752/1508] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:51:48.740672 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e681f-2b99-4235-aac9-74c1dedef822 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:51:48.741884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-de5e681f-2b99-4235-aac9-74c1dedef822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:51:48.839481 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e681f-2b99-4235-aac9-74c1dedef822 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:51:48.895383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:51:48.915652 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df8fdd-de3e-4665-97cc-73f986bcb148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:51:48.916286 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df8fdd-de3e-4665-97cc-73f986bcb148 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:48.916559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df8fdd-de3e-4665-97cc-73f986bcb148 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:48.916822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df8fdd-de3e-4665-97cc-73f986bcb148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:48.918406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f6df8fdd-de3e-4665-97cc-73f986bcb148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:51:49.023075 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df8fdd-de3e-4665-97cc-73f986bcb148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:51:49.024432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df8fdd-de3e-4665-97cc-73f986bcb148 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:51:49.043053 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 757/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:51:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:51:49.396764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8a65e-d8d8-4774-a464-b15306f1a631 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:51:49.398021 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9cf8a65e-d8d8-4774-a464-b15306f1a631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:51:49.409611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8a65e-d8d8-4774-a464-b15306f1a631 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:51:49.509909 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20ef21-7b61-48cc-b723-7793d044e3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:51:49.510541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20ef21-7b61-48cc-b723-7793d044e3de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:51:49.510815 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20ef21-7b61-48cc-b723-7793d044e3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:51:49.511082 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20ef21-7b61-48cc-b723-7793d044e3de admin admin] PATH_INFO: `/v3/projects/37c588e72d504829b6b7d34e0a823d20` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:51:49.521612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20ef21-7b61-48cc-b723-7793d044e3de admin admin] RBAC: Authorizing `identity:delete_project(project_id=37c588e72d504829b6b7d34e0a823d20)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:51:49.522641 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: WARNING py.warnings [None req-9b20ef21-7b61-48cc-b723-7793d044e3de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 07:51:49.522641 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: warnings.warn(msg) Sep 07 07:51:49.522641 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: Sep 07 07:51:49.523889 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20ef21-7b61-48cc-b723-7793d044e3de admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:51:49.580546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9b20ef21-7b61-48cc-b723-7793d044e3de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37c588e72d504829b6b7d34e0a823d20', 'request_id': 'req-9b20ef21-7b61-48cc-b723-7793d044e3de'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:51:49.586588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9b20ef21-7b61-48cc-b723-7793d044e3de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37c588e72d504829b6b7d34e0a823d20', 'request_id': 'req-9b20ef21-7b61-48cc-b723-7793d044e3de'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:51:49.599778 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9b20ef21-7b61-48cc-b723-7793d044e3de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37c588e72d504829b6b7d34e0a823d20', 'request_id': 'req-9b20ef21-7b61-48cc-b723-7793d044e3de'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:51:49.625606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9b20ef21-7b61-48cc-b723-7793d044e3de admin admin] The token cache is being invalidate because project 37c588e72d504829b6b7d34e0a823d20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:51:49.625899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9b20ef21-7b61-48cc-b723-7793d044e3de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:51:49.626327 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9b20ef21-7b61-48cc-b723-7793d044e3de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:51:49.627701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 753/1510] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:51:49 2022] DELETE /identity/v3/projects/37c588e72d504829b6b7d34e0a823d20 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:51:59.040762 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=62 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:51:59.623588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:53:14.429172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce2144-a06d-461a-b992-1805acbef5c7 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:14.429716 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce2144-a06d-461a-b992-1805acbef5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:14.429716 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce2144-a06d-461a-b992-1805acbef5c7 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:14.431106 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 07:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:53:14.436470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1606e54-7db2-4e2f-982d-b78fbccb764b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:14.436914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1606e54-7db2-4e2f-982d-b78fbccb764b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:14.437023 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1606e54-7db2-4e2f-982d-b78fbccb764b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:14.543538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e1606e54-7db2-4e2f-982d-b78fbccb764b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:53:14.544779 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e1606e54-7db2-4e2f-982d-b78fbccb764b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:53:14.647615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 07:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:53:14.805750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce2144-a06d-461a-b992-1805acbef5c7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:53:14.809561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-96ce2144-a06d-461a-b992-1805acbef5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:53:14.933232 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce2144-a06d-461a-b992-1805acbef5c7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:53:14.938564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efcd84-9a3c-44b7-b904-3e3d68d2a04f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:53:14.939162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efcd84-9a3c-44b7-b904-3e3d68d2a04f service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:14.939440 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efcd84-9a3c-44b7-b904-3e3d68d2a04f service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:14.939712 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efcd84-9a3c-44b7-b904-3e3d68d2a04f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:14.947898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41efcd84-9a3c-44b7-b904-3e3d68d2a04f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:53:14.949491 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41efcd84-9a3c-44b7-b904-3e3d68d2a04f service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:53:14.958678 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 759/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:53:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:53:16.044120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac92d78-be7a-45a2-8e6c-6724907b5c4b None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:16.044650 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac92d78-be7a-45a2-8e6c-6724907b5c4b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:16.044650 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac92d78-be7a-45a2-8e6c-6724907b5c4b None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:16.045922 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 07:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:53:16.051996 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873744-3326-4758-aff1-ddf113536dd4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:16.052423 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873744-3326-4758-aff1-ddf113536dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:16.052622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873744-3326-4758-aff1-ddf113536dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:16.072430 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f1873744-3326-4758-aff1-ddf113536dd4 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:53:16.073849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f1873744-3326-4758-aff1-ddf113536dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:53:16.081640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 07:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:53:16.200932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38759620-07ae-4973-93c8-16b7221446c8 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:16.201216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38759620-07ae-4973-93c8-16b7221446c8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:16.201492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38759620-07ae-4973-93c8-16b7221446c8 None None] PATH_INFO: `/v3` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:16.202767 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 756/1516] 10.222.0.27 () {54 vars in 925 bytes} [Wed Sep 7 07:53:16 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:53:16.209749 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ab4f8-fe03-4295-a640-0e9bf0599489 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:16.210043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ab4f8-fe03-4295-a640-0e9bf0599489 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:16.210351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ab4f8-fe03-4295-a640-0e9bf0599489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:16.239565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38759620-07ae-4973-93c8-16b7221446c8 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:53:16.255488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38759620-07ae-4973-93c8-16b7221446c8 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:53:16.261781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903c7c0-e9d7-46d9-8be2-cac0365cb4b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:53:16.262386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903c7c0-e9d7-46d9-8be2-cac0365cb4b7 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:16.262661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903c7c0-e9d7-46d9-8be2-cac0365cb4b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:16.262930 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903c7c0-e9d7-46d9-8be2-cac0365cb4b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:16.271335 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903c7c0-e9d7-46d9-8be2-cac0365cb4b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:53:16.272945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903c7c0-e9d7-46d9-8be2-cac0365cb4b7 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:53:16.278643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7b4ab4f8-fe03-4295-a640-0e9bf0599489 None None] MFA Rules not processed for user `b16b81ab30294bc4a8c87b6d7c3573f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:53:16.279862 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7b4ab4f8-fe03-4295-a640-0e9bf0599489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:53:16.283129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 757/1517] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:53:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:53:16.336714 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:53:16.358715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 761/1518] 10.222.0.27 () {58 vars in 1015 bytes} [Wed Sep 7 07:53:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:53:25.147830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf163af6-338f-4edc-8a2e-ec44c424af91 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:25.148325 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf163af6-338f-4edc-8a2e-ec44c424af91 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:25.148487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf163af6-338f-4edc-8a2e-ec44c424af91 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:25.149823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 758/1519] 10.222.0.26 () {54 vars in 961 bytes} [Wed Sep 7 07:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:53:25.155008 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e831ff-d8a9-4fe6-953b-17e8a8d89c12 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:25.155442 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e831ff-d8a9-4fe6-953b-17e8a8d89c12 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:25.155633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e831ff-d8a9-4fe6-953b-17e8a8d89c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:25.174863 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-57e831ff-d8a9-4fe6-953b-17e8a8d89c12 None None] MFA Rules not processed for user `b16b81ab30294bc4a8c87b6d7c3573f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:53:25.176045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-57e831ff-d8a9-4fe6-953b-17e8a8d89c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:53:25.185030 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 762/1520] 10.222.0.26 () {58 vars in 1058 bytes} [Wed Sep 7 07:53:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:53:25.192736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf163af6-338f-4edc-8a2e-ec44c424af91 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:53:25.206321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf163af6-338f-4edc-8a2e-ec44c424af91 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:53:25.211178 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d731ce-a60e-480b-8610-de8c9e8eced9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:53:25.211756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d731ce-a60e-480b-8610-de8c9e8eced9 service placement] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:25.212033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d731ce-a60e-480b-8610-de8c9e8eced9 service placement] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:25.212332 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d731ce-a60e-480b-8610-de8c9e8eced9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:25.220337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d731ce-a60e-480b-8610-de8c9e8eced9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:53:25.221881 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d731ce-a60e-480b-8610-de8c9e8eced9 service placement] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:53:25.231513 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 759/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:53:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:53:27.752824 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1972bb-6af6-4cf9-9b6d-1f34146cb009 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:27.753393 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1972bb-6af6-4cf9-9b6d-1f34146cb009 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:27.753393 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1972bb-6af6-4cf9-9b6d-1f34146cb009 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:27.754711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 07:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:53:27.761607 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f23052-e940-4c1b-921d-3161cf1a1b20 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:27.762045 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f23052-e940-4c1b-921d-3161cf1a1b20 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:27.762162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f23052-e940-4c1b-921d-3161cf1a1b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:27.781766 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f8f23052-e940-4c1b-921d-3161cf1a1b20 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:53:27.782955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f8f23052-e940-4c1b-921d-3161cf1a1b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:53:27.790693 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 07:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:53:27.944048 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1972bb-6af6-4cf9-9b6d-1f34146cb009 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:53:27.958443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1972bb-6af6-4cf9-9b6d-1f34146cb009 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:53:27.964431 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b9619-2138-474f-b19b-b4efc64a62d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:53:27.965012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b9619-2138-474f-b19b-b4efc64a62d1 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:27.965287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b9619-2138-474f-b19b-b4efc64a62d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:27.965554 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b9619-2138-474f-b19b-b4efc64a62d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:27.973394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b9619-2138-474f-b19b-b4efc64a62d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:53:27.974939 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b9619-2138-474f-b19b-b4efc64a62d1 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:53:27.984826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 764/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:53:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:53:29.052848 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e01eb-a351-40a2-aff7-398a59bb820d None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:29.053412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e01eb-a351-40a2-aff7-398a59bb820d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:29.053412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e01eb-a351-40a2-aff7-398a59bb820d None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:29.054746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 07:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:53:29.060934 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a48b93-b285-4a2f-b871-c26a9cf2082a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:29.061386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a48b93-b285-4a2f-b871-c26a9cf2082a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:29.061513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a48b93-b285-4a2f-b871-c26a9cf2082a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:29.081305 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c4a48b93-b285-4a2f-b871-c26a9cf2082a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:53:29.082492 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c4a48b93-b285-4a2f-b871-c26a9cf2082a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:53:29.091125 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 07:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:53:29.240759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e01eb-a351-40a2-aff7-398a59bb820d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:53:29.255671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e01eb-a351-40a2-aff7-398a59bb820d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:53:29.262336 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcfc1c-a4ed-498e-9bfe-aa24d00ea3ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:53:29.262935 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcfc1c-a4ed-498e-9bfe-aa24d00ea3ad service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:29.263216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcfc1c-a4ed-498e-9bfe-aa24d00ea3ad service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:29.263488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcfc1c-a4ed-498e-9bfe-aa24d00ea3ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:29.271602 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fcfc1c-a4ed-498e-9bfe-aa24d00ea3ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:53:29.273214 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fcfc1c-a4ed-498e-9bfe-aa24d00ea3ad service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:53:29.283473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 762/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:53:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:53:33.261111 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6301dcb-c40d-427f-b344-c9009c0a8159 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:33.261686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6301dcb-c40d-427f-b344-c9009c0a8159 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:33.261686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6301dcb-c40d-427f-b344-c9009c0a8159 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:33.262976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 07:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:53:33.268067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f998ae5-94ec-4947-b48b-5fc6fb308275 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:33.268530 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f998ae5-94ec-4947-b48b-5fc6fb308275 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:33.268655 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f998ae5-94ec-4947-b48b-5fc6fb308275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:33.288239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7f998ae5-94ec-4947-b48b-5fc6fb308275 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:53:33.289429 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7f998ae5-94ec-4947-b48b-5fc6fb308275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:53:33.297909 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 07:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:53:33.588572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6301dcb-c40d-427f-b344-c9009c0a8159 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:53:33.590274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a6301dcb-c40d-427f-b344-c9009c0a8159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:53:33.668073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6301dcb-c40d-427f-b344-c9009c0a8159 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:53:33.671683 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4abcd-f8ee-4883-8798-af46e30c56ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:53:33.672307 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4abcd-f8ee-4883-8798-af46e30c56ca service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:33.672586 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4abcd-f8ee-4883-8798-af46e30c56ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:33.672857 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4abcd-f8ee-4883-8798-af46e30c56ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:33.681873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4abcd-f8ee-4883-8798-af46e30c56ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:53:33.683285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4abcd-f8ee-4883-8798-af46e30c56ca service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:53:33.692799 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 767/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:53:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:53:35.866446 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427d1e3-e6d3-4448-b5e9-4d3d1993027a None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:35.867002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427d1e3-e6d3-4448-b5e9-4d3d1993027a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:35.867002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427d1e3-e6d3-4448-b5e9-4d3d1993027a None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:35.868423 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 07:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:53:35.874209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067eba5-d9dd-44a1-83ec-1386c515d18d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:35.874716 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067eba5-d9dd-44a1-83ec-1386c515d18d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:35.874824 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067eba5-d9dd-44a1-83ec-1386c515d18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:35.894973 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f067eba5-d9dd-44a1-83ec-1386c515d18d None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:53:35.896133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f067eba5-d9dd-44a1-83ec-1386c515d18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:53:35.904595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 07:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:53:36.191725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427d1e3-e6d3-4448-b5e9-4d3d1993027a None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:53:36.193671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c427d1e3-e6d3-4448-b5e9-4d3d1993027a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:53:36.206823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427d1e3-e6d3-4448-b5e9-4d3d1993027a None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:53:36.210507 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9ae7c-b35c-4b30-aafa-a39299e3f73d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:53:36.211104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9ae7c-b35c-4b30-aafa-a39299e3f73d service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:36.211384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9ae7c-b35c-4b30-aafa-a39299e3f73d service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:36.211657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9ae7c-b35c-4b30-aafa-a39299e3f73d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:36.220776 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9ae7c-b35c-4b30-aafa-a39299e3f73d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:53:36.222200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9ae7c-b35c-4b30-aafa-a39299e3f73d service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:53:36.231482 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 765/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:53:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:53:39.772656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a92ef-1c9d-4824-9d16-0305621fcc73 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:39.773240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a92ef-1c9d-4824-9d16-0305621fcc73 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:39.773240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a92ef-1c9d-4824-9d16-0305621fcc73 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:39.774558 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 07:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:53:39.780311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed37e3-b064-414a-9c9a-2b933f8e80a6 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:39.780746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed37e3-b064-414a-9c9a-2b933f8e80a6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:39.780864 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed37e3-b064-414a-9c9a-2b933f8e80a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:39.788439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:53:39.801550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-78ed37e3-b064-414a-9c9a-2b933f8e80a6 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:53:39.802739 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-78ed37e3-b064-414a-9c9a-2b933f8e80a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:53:39.811174 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 07:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:53:40.118251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a92ef-1c9d-4824-9d16-0305621fcc73 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:53:40.120681 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d74a92ef-1c9d-4824-9d16-0305621fcc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:53:40.198037 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a92ef-1c9d-4824-9d16-0305621fcc73 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:53:40.201584 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25f25d-ddf8-418f-ba99-1085f00bf7e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:53:40.202181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25f25d-ddf8-418f-ba99-1085f00bf7e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:40.202458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25f25d-ddf8-418f-ba99-1085f00bf7e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:40.202730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25f25d-ddf8-418f-ba99-1085f00bf7e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:40.211416 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25f25d-ddf8-418f-ba99-1085f00bf7e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:53:40.213150 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25f25d-ddf8-418f-ba99-1085f00bf7e4 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:53:40.222311 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 770/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:53:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:53:42.350262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181b681-cfcd-4796-83fe-0fafedcf5696 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:42.350829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181b681-cfcd-4796-83fe-0fafedcf5696 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:42.350829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181b681-cfcd-4796-83fe-0fafedcf5696 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:42.352216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 07:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:53:42.357422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec93d8c-b303-4fc3-837c-13f4ab894d24 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:42.357877 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec93d8c-b303-4fc3-837c-13f4ab894d24 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:42.357985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec93d8c-b303-4fc3-837c-13f4ab894d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:42.378011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3ec93d8c-b303-4fc3-837c-13f4ab894d24 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:53:42.379238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3ec93d8c-b303-4fc3-837c-13f4ab894d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:53:42.388097 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 07:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:53:42.689303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181b681-cfcd-4796-83fe-0fafedcf5696 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:53:42.690996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a181b681-cfcd-4796-83fe-0fafedcf5696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:53:42.704415 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181b681-cfcd-4796-83fe-0fafedcf5696 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:53:42.707834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b715b22-36f2-42be-8a39-6fedaa6ea760 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:53:42.708458 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b715b22-36f2-42be-8a39-6fedaa6ea760 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:53:42.708735 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b715b22-36f2-42be-8a39-6fedaa6ea760 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:53:42.709003 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b715b22-36f2-42be-8a39-6fedaa6ea760 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:53:42.717605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b715b22-36f2-42be-8a39-6fedaa6ea760 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:53:42.719008 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b715b22-36f2-42be-8a39-6fedaa6ea760 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:53:42.728880 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 768/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:53:52.373138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:53:52.724141 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=8 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:06.075194 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e0f4d-a878-4e39-9ffd-7f5c2027c02b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.076464 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e0f4d-a878-4e39-9ffd-7f5c2027c02b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.076464 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e0f4d-a878-4e39-9ffd-7f5c2027c02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.112466 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-219e0f4d-a878-4e39-9ffd-7f5c2027c02b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:06.113852 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-219e0f4d-a878-4e39-9ffd-7f5c2027c02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:06.127049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 772/1540] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:55:06.136658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce13501-93e2-4e97-8d85-79814a3619d1 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.137633 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce13501-93e2-4e97-8d85-79814a3619d1 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.138110 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce13501-93e2-4e97-8d85-79814a3619d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.145322 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b6f2d-cf56-428f-b1e3-abd69305ad69 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.145621 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b6f2d-cf56-428f-b1e3-abd69305ad69 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.145883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b6f2d-cf56-428f-b1e3-abd69305ad69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.172624 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-2ce13501-93e2-4e97-8d85-79814a3619d1 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:06.174020 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2ce13501-93e2-4e97-8d85-79814a3619d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:06.174967 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ba5b6f2d-cf56-428f-b1e3-abd69305ad69 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:06.176312 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ba5b6f2d-cf56-428f-b1e3-abd69305ad69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:06.191248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 769/1541] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:06.196067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce13501-93e2-4e97-8d85-79814a3619d1 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:06.198215 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 773/1542] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:55:06.212056 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce13501-93e2-4e97-8d85-79814a3619d1 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:06.217416 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d260120-49a4-4378-ad7f-c9b4223d8efb None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.217812 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d260120-49a4-4378-ad7f-c9b4223d8efb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.218204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d260120-49a4-4378-ad7f-c9b4223d8efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.225115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974aceb-2420-4642-88cc-c704661c9db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:06.225737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974aceb-2420-4642-88cc-c704661c9db0 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.226012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974aceb-2420-4642-88cc-c704661c9db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.226281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974aceb-2420-4642-88cc-c704661c9db0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.226916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974aceb-2420-4642-88cc-c704661c9db0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:06.227212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974aceb-2420-4642-88cc-c704661c9db0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:06.228933 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974aceb-2420-4642-88cc-c704661c9db0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:06.248238 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 770/1543] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:06.249643 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab3e1f-4c07-4c9b-941f-6b9d52312203 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.249929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab3e1f-4c07-4c9b-941f-6b9d52312203 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.250201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab3e1f-4c07-4c9b-941f-6b9d52312203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.253295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6d260120-49a4-4378-ad7f-c9b4223d8efb None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:06.254569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6d260120-49a4-4378-ad7f-c9b4223d8efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:06.266421 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 774/1544] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:06.267259 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d260120-49a4-4378-ad7f-c9b4223d8efb None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:06.283610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d260120-49a4-4378-ad7f-c9b4223d8efb None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:06.288931 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7fab3e1f-4c07-4c9b-941f-6b9d52312203 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:06.290152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7fab3e1f-4c07-4c9b-941f-6b9d52312203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:06.293210 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8c085-bb8b-4b44-8fbd-8af7cd70db17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:06.293858 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8c085-bb8b-4b44-8fbd-8af7cd70db17 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.294147 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8c085-bb8b-4b44-8fbd-8af7cd70db17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.294431 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8c085-bb8b-4b44-8fbd-8af7cd70db17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.295086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8c085-bb8b-4b44-8fbd-8af7cd70db17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:06.299251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8c085-bb8b-4b44-8fbd-8af7cd70db17 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:06.319647 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 771/1545] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:55:06.320623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab3e1f-4c07-4c9b-941f-6b9d52312203 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:06.342880 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab3e1f-4c07-4c9b-941f-6b9d52312203 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:06.357678 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d5993-5292-46ae-8da4-a57a88b79876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:06.359576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 775/1546] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:06.361613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d5993-5292-46ae-8da4-a57a88b79876 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.363532 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5637f8-a215-4bcf-b1ca-68b830f6b1de None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.363804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d5993-5292-46ae-8da4-a57a88b79876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.364360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d5993-5292-46ae-8da4-a57a88b79876 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.365236 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d5993-5292-46ae-8da4-a57a88b79876 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:06.365841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5637f8-a215-4bcf-b1ca-68b830f6b1de None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.366319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5637f8-a215-4bcf-b1ca-68b830f6b1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.369482 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d5993-5292-46ae-8da4-a57a88b79876 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:06.371468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d5993-5292-46ae-8da4-a57a88b79876 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:06.401201 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6b5637f8-a215-4bcf-b1ca-68b830f6b1de None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:06.402501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 772/1547] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:06.402733 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6b5637f8-a215-4bcf-b1ca-68b830f6b1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:06.405062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151a881-74a2-469c-b1e6-ebf78395edc4 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.405582 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151a881-74a2-469c-b1e6-ebf78395edc4 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.406221 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151a881-74a2-469c-b1e6-ebf78395edc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.437339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1151a881-74a2-469c-b1e6-ebf78395edc4 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:06.438530 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1151a881-74a2-469c-b1e6-ebf78395edc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:06.483106 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 776/1548] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:06.485773 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed52996-9a3e-45b2-8bfe-000b6d39ef46 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.486273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed52996-9a3e-45b2-8bfe-000b6d39ef46 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.486850 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed52996-9a3e-45b2-8bfe-000b6d39ef46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.510944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 773/1549] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:06.512039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151a881-74a2-469c-b1e6-ebf78395edc4 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:06.512929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-aed52996-9a3e-45b2-8bfe-000b6d39ef46 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:06.514174 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-aed52996-9a3e-45b2-8bfe-000b6d39ef46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:06.521255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151a881-74a2-469c-b1e6-ebf78395edc4 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:06.522661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 777/1550] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:06.523865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fca25-2594-4d85-aa10-868381f3c4af None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.524194 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fca25-2594-4d85-aa10-868381f3c4af None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.524473 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fca25-2594-4d85-aa10-868381f3c4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.531520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895902bc-f0c9-4b6e-b7ef-814e54e8063a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:06.532145 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895902bc-f0c9-4b6e-b7ef-814e54e8063a admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.532421 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895902bc-f0c9-4b6e-b7ef-814e54e8063a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.532901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895902bc-f0c9-4b6e-b7ef-814e54e8063a admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.534399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895902bc-f0c9-4b6e-b7ef-814e54e8063a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:06.536386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895902bc-f0c9-4b6e-b7ef-814e54e8063a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:06.544938 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3b8fca25-2594-4d85-aa10-868381f3c4af None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:06.546153 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3b8fca25-2594-4d85-aa10-868381f3c4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:06.554654 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 778/1551] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:06.555410 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fca25-2594-4d85-aa10-868381f3c4af None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:06.562615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 774/1552] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/users => generated 288 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:06.562615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94254257-7e34-4eb1-adec-779404f13f23 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.562615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94254257-7e34-4eb1-adec-779404f13f23 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.562615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94254257-7e34-4eb1-adec-779404f13f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.566055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fca25-2594-4d85-aa10-868381f3c4af None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:06.570998 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a6b19-cca8-490f-82b5-6d039f3a2a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:06.571595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a6b19-cca8-490f-82b5-6d039f3a2a44 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.571948 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a6b19-cca8-490f-82b5-6d039f3a2a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.572361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a6b19-cca8-490f-82b5-6d039f3a2a44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.573534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a6b19-cca8-490f-82b5-6d039f3a2a44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:06.575733 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a6b19-cca8-490f-82b5-6d039f3a2a44 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:06.581968 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-94254257-7e34-4eb1-adec-779404f13f23 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:06.583158 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-94254257-7e34-4eb1-adec-779404f13f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:06.591058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 775/1553] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:06.592250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd69c2-2e92-4206-b12f-c455f014325e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.592521 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd69c2-2e92-4206-b12f-c455f014325e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.593311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd69c2-2e92-4206-b12f-c455f014325e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.608114 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 779/1554] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:06.608972 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bf16f-e3a0-4c64-b644-eca1941bd677 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.609263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bf16f-e3a0-4c64-b644-eca1941bd677 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.609536 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bf16f-e3a0-4c64-b644-eca1941bd677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.616170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7cfd69c2-2e92-4206-b12f-c455f014325e None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:06.617350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7cfd69c2-2e92-4206-b12f-c455f014325e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:06.629012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cd9bf16f-e3a0-4c64-b644-eca1941bd677 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:06.630206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cd9bf16f-e3a0-4c64-b644-eca1941bd677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:06.677674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 776/1555] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:06.678552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd69c2-2e92-4206-b12f-c455f014325e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:06.688196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd69c2-2e92-4206-b12f-c455f014325e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:06.689777 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 780/1556] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:06.691074 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8a1a3-beb7-4f7b-97e4-18995278a7b7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.691359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8a1a3-beb7-4f7b-97e4-18995278a7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.691627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8a1a3-beb7-4f7b-97e4-18995278a7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.700605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e86ce-8eee-4e21-a3de-5b5673186763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:06.701216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e86ce-8eee-4e21-a3de-5b5673186763 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.701490 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e86ce-8eee-4e21-a3de-5b5673186763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.701755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e86ce-8eee-4e21-a3de-5b5673186763 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.702384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e86ce-8eee-4e21-a3de-5b5673186763 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:06.702679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e86ce-8eee-4e21-a3de-5b5673186763 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:06.704313 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e86ce-8eee-4e21-a3de-5b5673186763 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:06.711424 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c5c8a1a3-beb7-4f7b-97e4-18995278a7b7 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:06.713102 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c5c8a1a3-beb7-4f7b-97e4-18995278a7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:06.716638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 777/1557] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:06.717287 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e86ce-8eee-4e21-a3de-5b5673186763 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:06.724195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 781/1558] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:06.725689 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30cab3-8c7f-4da0-96a8-44d1f7b4a658 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.725980 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30cab3-8c7f-4da0-96a8-44d1f7b4a658 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.726252 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30cab3-8c7f-4da0-96a8-44d1f7b4a658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.729710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e86ce-8eee-4e21-a3de-5b5673186763 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:06.734414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6787de4-5429-4cc4-aee9-4b58f24f21f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:06.734997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6787de4-5429-4cc4-aee9-4b58f24f21f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.735276 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6787de4-5429-4cc4-aee9-4b58f24f21f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.735547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6787de4-5429-4cc4-aee9-4b58f24f21f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.736220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6787de4-5429-4cc4-aee9-4b58f24f21f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:06.737808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6787de4-5429-4cc4-aee9-4b58f24f21f7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:06.745946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9e30cab3-8c7f-4da0-96a8-44d1f7b4a658 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:06.747125 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9e30cab3-8c7f-4da0-96a8-44d1f7b4a658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:06.748237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 778/1559] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:06.749400 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13b9e5-a0d1-42c0-bacf-aabfb766972b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.749748 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13b9e5-a0d1-42c0-bacf-aabfb766972b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.749933 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13b9e5-a0d1-42c0-bacf-aabfb766972b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.755235 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 782/1560] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:06.756035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30cab3-8c7f-4da0-96a8-44d1f7b4a658 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:06.761461 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:06.766367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30cab3-8c7f-4da0-96a8-44d1f7b4a658 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:06.772577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a02785-2757-457f-b239-7b7729802268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:06.772962 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-eb13b9e5-a0d1-42c0-bacf-aabfb766972b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:06.773170 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a02785-2757-457f-b239-7b7729802268 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.773448 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a02785-2757-457f-b239-7b7729802268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.773718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a02785-2757-457f-b239-7b7729802268 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.774143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-eb13b9e5-a0d1-42c0-bacf-aabfb766972b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:06.774339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a02785-2757-457f-b239-7b7729802268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:06.775509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-56a02785-2757-457f-b239-7b7729802268 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 07:55:06.775509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Sep 07 07:55:06.775509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: Sep 07 07:55:06.776742 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a02785-2757-457f-b239-7b7729802268 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:06.782143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 779/1561] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:06.782973 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13b9e5-a0d1-42c0-bacf-aabfb766972b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:06.793808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13b9e5-a0d1-42c0-bacf-aabfb766972b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:06.798443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aee4df-b851-4f51-987b-00343403f59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:06.799027 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aee4df-b851-4f51-987b-00343403f59c admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.799298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aee4df-b851-4f51-987b-00343403f59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.799565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aee4df-b851-4f51-987b-00343403f59c admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.800445 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aee4df-b851-4f51-987b-00343403f59c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:06.800571 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 783/1562] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/users => generated 282 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:06.800668 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aee4df-b851-4f51-987b-00343403f59c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:06.801750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972b5c8-8ae2-4ef2-8775-09252bf62fe4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.802181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972b5c8-8ae2-4ef2-8775-09252bf62fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.802298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aee4df-b851-4f51-987b-00343403f59c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:06.802462 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972b5c8-8ae2-4ef2-8775-09252bf62fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.803933 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:06.814937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 780/1563] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:06.816638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aee4df-b851-4f51-987b-00343403f59c admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:06.823776 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4972b5c8-8ae2-4ef2-8775-09252bf62fe4 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:06.825037 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4972b5c8-8ae2-4ef2-8775-09252bf62fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:06.827308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aee4df-b851-4f51-987b-00343403f59c admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:06.831796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d13b49-5836-4bf7-81be-296e0b295d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:06.832514 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d13b49-5836-4bf7-81be-296e0b295d96 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.832787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d13b49-5836-4bf7-81be-296e0b295d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.833147 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 784/1564] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:06.833277 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d13b49-5836-4bf7-81be-296e0b295d96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.833788 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d13b49-5836-4bf7-81be-296e0b295d96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:06.833901 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972b5c8-8ae2-4ef2-8775-09252bf62fe4 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:06.834830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: WARNING py.warnings [None req-82d13b49-5836-4bf7-81be-296e0b295d96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 07:55:06.834830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: warnings.warn(msg) Sep 07 07:55:06.834830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: Sep 07 07:55:06.835997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d13b49-5836-4bf7-81be-296e0b295d96 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:06.844666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972b5c8-8ae2-4ef2-8775-09252bf62fe4 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:06.849315 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dac6fb-f58c-4429-92d3-f9549e43594a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:06.849905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dac6fb-f58c-4429-92d3-f9549e43594a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.850185 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dac6fb-f58c-4429-92d3-f9549e43594a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.850459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dac6fb-f58c-4429-92d3-f9549e43594a admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.851090 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dac6fb-f58c-4429-92d3-f9549e43594a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:06.851392 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dac6fb-f58c-4429-92d3-f9549e43594a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:06.852428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-b8dac6fb-f58c-4429-92d3-f9549e43594a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 07:55:06.852428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Sep 07 07:55:06.852428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: Sep 07 07:55:06.853469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dac6fb-f58c-4429-92d3-f9549e43594a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:06.865743 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 785/1565] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:06.869771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 781/1566] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:06.869907 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dac6fb-f58c-4429-92d3-f9549e43594a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:06.869994 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d13b49-5836-4bf7-81be-296e0b295d96 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:06.880205 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dac6fb-f58c-4429-92d3-f9549e43594a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:06.936202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748e980-eb47-4fef-a7a5-079e4eaba22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:06.936429 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d13b49-5836-4bf7-81be-296e0b295d96 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:06.936807 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748e980-eb47-4fef-a7a5-079e4eaba22f admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.937082 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748e980-eb47-4fef-a7a5-079e4eaba22f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.937463 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748e980-eb47-4fef-a7a5-079e4eaba22f admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.938099 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748e980-eb47-4fef-a7a5-079e4eaba22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:06.939673 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748e980-eb47-4fef-a7a5-079e4eaba22f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:06.941141 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4fcbb-edfb-4a1d-9fa9-f6bc8248c8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:06.941782 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4fcbb-edfb-4a1d-9fa9-f6bc8248c8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.942343 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4fcbb-edfb-4a1d-9fa9-f6bc8248c8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.942613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4fcbb-edfb-4a1d-9fa9-f6bc8248c8f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.943270 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4fcbb-edfb-4a1d-9fa9-f6bc8248c8f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:06.943571 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4fcbb-edfb-4a1d-9fa9-f6bc8248c8f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:06.945212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4fcbb-edfb-4a1d-9fa9-f6bc8248c8f7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:06.953738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 786/1567] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:06.953738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748e980-eb47-4fef-a7a5-079e4eaba22f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:06.959158 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 782/1568] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:06.960050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4fcbb-edfb-4a1d-9fa9-f6bc8248c8f7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:06.968931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748e980-eb47-4fef-a7a5-079e4eaba22f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:06.970378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4fcbb-edfb-4a1d-9fa9-f6bc8248c8f7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:06.973576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8c1bf-24c4-4683-b100-c6c80331781c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:06.974271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8c1bf-24c4-4683-b100-c6c80331781c admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.974550 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8c1bf-24c4-4683-b100-c6c80331781c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.974821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8c1bf-24c4-4683-b100-c6c80331781c admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.975221 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35aa5e4-1284-4bd6-8bcb-b32408b35b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:06.975453 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8c1bf-24c4-4683-b100-c6c80331781c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:06.975751 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8c1bf-24c4-4683-b100-c6c80331781c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:06.976006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35aa5e4-1284-4bd6-8bcb-b32408b35b78 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:06.976372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35aa5e4-1284-4bd6-8bcb-b32408b35b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:06.976731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35aa5e4-1284-4bd6-8bcb-b32408b35b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:06.977364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8c1bf-24c4-4683-b100-c6c80331781c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:06.977461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35aa5e4-1284-4bd6-8bcb-b32408b35b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:06.979956 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35aa5e4-1284-4bd6-8bcb-b32408b35b78 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:06.990436 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 787/1569] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:06.993797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 783/1570] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:06.993938 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8c1bf-24c4-4683-b100-c6c80331781c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:06.994043 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35aa5e4-1284-4bd6-8bcb-b32408b35b78 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.003285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35aa5e4-1284-4bd6-8bcb-b32408b35b78 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.007128 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8c1bf-24c4-4683-b100-c6c80331781c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.007920 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e17fc-aba6-499d-9939-cb18238eca8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.008541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e17fc-aba6-499d-9939-cb18238eca8b admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.008818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e17fc-aba6-499d-9939-cb18238eca8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.009085 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e17fc-aba6-499d-9939-cb18238eca8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.010138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e17fc-aba6-499d-9939-cb18238eca8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:07.010438 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e17fc-aba6-499d-9939-cb18238eca8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.012015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e17fc-aba6-499d-9939-cb18238eca8b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.012143 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a79a1b-82e6-46a8-9680-27f483724101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.012691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a79a1b-82e6-46a8-9680-27f483724101 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.012966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a79a1b-82e6-46a8-9680-27f483724101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.013262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a79a1b-82e6-46a8-9680-27f483724101 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.013885 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a79a1b-82e6-46a8-9680-27f483724101 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.015834 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a79a1b-82e6-46a8-9680-27f483724101 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.023490 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 784/1571] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:07.024294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e17fc-aba6-499d-9939-cb18238eca8b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.036515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e17fc-aba6-499d-9939-cb18238eca8b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.042241 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330df534-b656-43fe-aca0-720c39587dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.042830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330df534-b656-43fe-aca0-720c39587dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.043105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330df534-b656-43fe-aca0-720c39587dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.043376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330df534-b656-43fe-aca0-720c39587dc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.043992 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330df534-b656-43fe-aca0-720c39587dc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.045135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 788/1572] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:06 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:07.045926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a79a1b-82e6-46a8-9680-27f483724101 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.049040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330df534-b656-43fe-aca0-720c39587dc6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.070878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 785/1573] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:07 2022] POST /identity/v3/users => generated 286 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:07.071655 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330df534-b656-43fe-aca0-720c39587dc6 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.080818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330df534-b656-43fe-aca0-720c39587dc6 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.114329 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a79a1b-82e6-46a8-9680-27f483724101 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.119309 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c235b-1018-4e6f-bb4e-59cd7a0026a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.119920 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c235b-1018-4e6f-bb4e-59cd7a0026a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.120225 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c235b-1018-4e6f-bb4e-59cd7a0026a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.120493 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c235b-1018-4e6f-bb4e-59cd7a0026a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.121117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c235b-1018-4e6f-bb4e-59cd7a0026a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.123003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c235b-1018-4e6f-bb4e-59cd7a0026a0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.136943 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da22495-111f-4849-af3a-48e536481a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.137603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da22495-111f-4849-af3a-48e536481a18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.137876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da22495-111f-4849-af3a-48e536481a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.138146 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da22495-111f-4849-af3a-48e536481a18 admin admin] PATH_INFO: `/v3/projects/94ef8ab5782f4d4cb8ceb6d8b78911ba/users/04e229ddb87d4e6eaff2fd433ec2606c/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.151396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 789/1574] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:07 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:07.154881 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c235b-1018-4e6f-bb4e-59cd7a0026a0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.154982 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da22495-111f-4849-af3a-48e536481a18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94ef8ab5782f4d4cb8ceb6d8b78911ba, user_id=04e229ddb87d4e6eaff2fd433ec2606c, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.155509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: WARNING py.warnings [None req-2da22495-111f-4849-af3a-48e536481a18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 07:55:07.155509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: warnings.warn(msg) Sep 07 07:55:07.155509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: Sep 07 07:55:07.157656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da22495-111f-4849-af3a-48e536481a18 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.164560 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:07.171999 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 786/1575] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:07 2022] PUT /identity/v3/projects/94ef8ab5782f4d4cb8ceb6d8b78911ba/users/04e229ddb87d4e6eaff2fd433ec2606c/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:07.172831 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da22495-111f-4849-af3a-48e536481a18 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.192264 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:07.217016 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c235b-1018-4e6f-bb4e-59cd7a0026a0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.248085 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da22495-111f-4849-af3a-48e536481a18 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.253042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0d005-4cbf-465c-9cd4-564c62b8ed0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.253637 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0d005-4cbf-465c-9cd4-564c62b8ed0a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.253914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0d005-4cbf-465c-9cd4-564c62b8ed0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.254180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0d005-4cbf-465c-9cd4-564c62b8ed0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.254804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0d005-4cbf-465c-9cd4-564c62b8ed0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.256434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0d005-4cbf-465c-9cd4-564c62b8ed0a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.266888 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 787/1576] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:07.267734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0d005-4cbf-465c-9cd4-564c62b8ed0a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.272379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaceecb-4edd-42c8-80e9-0312c7bca84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.272981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaceecb-4edd-42c8-80e9-0312c7bca84c admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.273261 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaceecb-4edd-42c8-80e9-0312c7bca84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.273535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaceecb-4edd-42c8-80e9-0312c7bca84c admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.274165 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaceecb-4edd-42c8-80e9-0312c7bca84c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.277007 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaceecb-4edd-42c8-80e9-0312c7bca84c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.277963 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0d005-4cbf-465c-9cd4-564c62b8ed0a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.282412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a9151-3d5f-4d40-8e58-4c9698833146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.282986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a9151-3d5f-4d40-8e58-4c9698833146 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.283262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a9151-3d5f-4d40-8e58-4c9698833146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.283530 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a9151-3d5f-4d40-8e58-4c9698833146 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.284164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a9151-3d5f-4d40-8e58-4c9698833146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.286028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a9151-3d5f-4d40-8e58-4c9698833146 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.305659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 790/1577] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:07 2022] POST /identity/v3/projects => generated 389 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:07.306496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaceecb-4edd-42c8-80e9-0312c7bca84c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.317424 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 788/1578] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:07 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:07.318212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a9151-3d5f-4d40-8e58-4c9698833146 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.369685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaceecb-4edd-42c8-80e9-0312c7bca84c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.382021 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a9151-3d5f-4d40-8e58-4c9698833146 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.387876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c81ad-5b79-4813-aaf4-ca4b65ae6bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.388515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c81ad-5b79-4813-aaf4-ca4b65ae6bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.388794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c81ad-5b79-4813-aaf4-ca4b65ae6bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.389063 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c81ad-5b79-4813-aaf4-ca4b65ae6bc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.389687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c81ad-5b79-4813-aaf4-ca4b65ae6bc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.391741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c81ad-5b79-4813-aaf4-ca4b65ae6bc6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.414123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 789/1579] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:07 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:07.418162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c81ad-5b79-4813-aaf4-ca4b65ae6bc6 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.422567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:07.427924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c81ad-5b79-4813-aaf4-ca4b65ae6bc6 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.432772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4344b1b-09ab-44a6-b23d-d2ace2128a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.433360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4344b1b-09ab-44a6-b23d-d2ace2128a38 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.433632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4344b1b-09ab-44a6-b23d-d2ace2128a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.433883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97177e2b-cb54-4f84-b483-f2c69801468b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.434019 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4344b1b-09ab-44a6-b23d-d2ace2128a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.434490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97177e2b-cb54-4f84-b483-f2c69801468b admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.434630 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4344b1b-09ab-44a6-b23d-d2ace2128a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.434766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97177e2b-cb54-4f84-b483-f2c69801468b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.435041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97177e2b-cb54-4f84-b483-f2c69801468b admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.435671 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97177e2b-cb54-4f84-b483-f2c69801468b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.436527 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4344b1b-09ab-44a6-b23d-d2ace2128a38 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.437608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97177e2b-cb54-4f84-b483-f2c69801468b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.447381 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 790/1580] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:07.448240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4344b1b-09ab-44a6-b23d-d2ace2128a38 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.457568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4344b1b-09ab-44a6-b23d-d2ace2128a38 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.463101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2076c3-0a48-48fc-a1f1-f52f30b43511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.463878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2076c3-0a48-48fc-a1f1-f52f30b43511 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.464260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2076c3-0a48-48fc-a1f1-f52f30b43511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.464636 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2076c3-0a48-48fc-a1f1-f52f30b43511 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.465329 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2076c3-0a48-48fc-a1f1-f52f30b43511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.469860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 791/1581] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:07 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:07.470025 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2076c3-0a48-48fc-a1f1-f52f30b43511 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.470128 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ad3f2-3b4b-496a-95c0-7f1e5d8cf7e0 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.470128 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ad3f2-3b4b-496a-95c0-7f1e5d8cf7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.470128 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ad3f2-3b4b-496a-95c0-7f1e5d8cf7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.489515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 791/1582] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:07 2022] POST /identity/v3/users => generated 292 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:07.490396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2076c3-0a48-48fc-a1f1-f52f30b43511 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.500177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2076c3-0a48-48fc-a1f1-f52f30b43511 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.500843 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1a1ad3f2-3b4b-496a-95c0-7f1e5d8cf7e0 None None] MFA Rules not processed for user `04e229ddb87d4e6eaff2fd433ec2606c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:07.502122 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1a1ad3f2-3b4b-496a-95c0-7f1e5d8cf7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:07.540827 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:07.559270 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f49af1-d253-4664-a01c-ef67c4063e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.559925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f49af1-d253-4664-a01c-ef67c4063e31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.560246 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f49af1-d253-4664-a01c-ef67c4063e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.560519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f49af1-d253-4664-a01c-ef67c4063e31 admin admin] PATH_INFO: `/v3/projects/1ded590044ea49dfba66ab456d831fe1/users/0778bbd644bf4b2ea70a8031ebb12783/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.566717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 792/1583] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:07.567894 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ad3f2-3b4b-496a-95c0-7f1e5d8cf7e0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.576469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f49af1-d253-4664-a01c-ef67c4063e31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ded590044ea49dfba66ab456d831fe1, user_id=0778bbd644bf4b2ea70a8031ebb12783, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.579274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f49af1-d253-4664-a01c-ef67c4063e31 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.579435 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ad3f2-3b4b-496a-95c0-7f1e5d8cf7e0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.584165 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b454b3d-ff7f-4718-96f2-7d84cda3105f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.585126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b454b3d-ff7f-4718-96f2-7d84cda3105f admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.585285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b454b3d-ff7f-4718-96f2-7d84cda3105f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.585740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b454b3d-ff7f-4718-96f2-7d84cda3105f admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.586454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b454b3d-ff7f-4718-96f2-7d84cda3105f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.588970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b454b3d-ff7f-4718-96f2-7d84cda3105f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.593622 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 792/1584] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:07 2022] PUT /identity/v3/projects/1ded590044ea49dfba66ab456d831fe1/users/0778bbd644bf4b2ea70a8031ebb12783/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:07.594424 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f49af1-d253-4664-a01c-ef67c4063e31 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.628700 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 793/1585] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:07 2022] POST /identity/v3/users => generated 295 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:07.628700 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b454b3d-ff7f-4718-96f2-7d84cda3105f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.637031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b454b3d-ff7f-4718-96f2-7d84cda3105f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.663924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f49af1-d253-4664-a01c-ef67c4063e31 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.670275 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40914d03-1a05-4345-b0be-d62c8d7e14ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.670882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40914d03-1a05-4345-b0be-d62c8d7e14ae admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.671159 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40914d03-1a05-4345-b0be-d62c8d7e14ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.671431 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40914d03-1a05-4345-b0be-d62c8d7e14ae admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.672064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40914d03-1a05-4345-b0be-d62c8d7e14ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.674011 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40914d03-1a05-4345-b0be-d62c8d7e14ae admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.696427 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40a501-de2f-4fd3-9575-a50c8726e60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.696665 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 793/1586] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:07 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:07.698040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40914d03-1a05-4345-b0be-d62c8d7e14ae admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.698938 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40a501-de2f-4fd3-9575-a50c8726e60a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.699283 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40a501-de2f-4fd3-9575-a50c8726e60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.699632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40a501-de2f-4fd3-9575-a50c8726e60a admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.700804 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc40a501-de2f-4fd3-9575-a50c8726e60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.703349 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc40a501-de2f-4fd3-9575-a50c8726e60a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.709324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40914d03-1a05-4345-b0be-d62c8d7e14ae admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.716231 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 794/1587] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:07.716231 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40a501-de2f-4fd3-9575-a50c8726e60a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.719449 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f5c98-2e65-4492-a031-b53273aa2097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.720060 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f5c98-2e65-4492-a031-b53273aa2097 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.720385 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f5c98-2e65-4492-a031-b53273aa2097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.720660 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f5c98-2e65-4492-a031-b53273aa2097 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.721299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f5c98-2e65-4492-a031-b53273aa2097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.723049 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f5c98-2e65-4492-a031-b53273aa2097 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.723801 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40a501-de2f-4fd3-9575-a50c8726e60a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.728566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395dd059-4a69-4669-8e1b-fd2f79377664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.729196 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395dd059-4a69-4669-8e1b-fd2f79377664 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.729476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395dd059-4a69-4669-8e1b-fd2f79377664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.729748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395dd059-4a69-4669-8e1b-fd2f79377664 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.730403 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395dd059-4a69-4669-8e1b-fd2f79377664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.732366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395dd059-4a69-4669-8e1b-fd2f79377664 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.734356 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 794/1588] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:07.735083 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f5c98-2e65-4492-a031-b53273aa2097 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.746856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f5c98-2e65-4492-a031-b53273aa2097 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.751633 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9f1a2-1dcc-4aa9-879e-8c1a8b104c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.752285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9f1a2-1dcc-4aa9-879e-8c1a8b104c78 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.752565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9f1a2-1dcc-4aa9-879e-8c1a8b104c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.752836 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9f1a2-1dcc-4aa9-879e-8c1a8b104c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.753469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9f1a2-1dcc-4aa9-879e-8c1a8b104c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.754335 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 795/1589] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:07 2022] POST /identity/v3/users => generated 294 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:07.755059 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9f1a2-1dcc-4aa9-879e-8c1a8b104c78 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.758584 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d44f8-495d-4e98-bcd4-217306cdac9d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.758876 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d44f8-495d-4e98-bcd4-217306cdac9d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.759153 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d44f8-495d-4e98-bcd4-217306cdac9d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.761230 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 796/1590] 10.222.0.26 () {54 vars in 962 bytes} [Wed Sep 7 07:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:55:07.762292 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc5b9d-cba7-4d56-8366-874fdc649e8d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.762576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc5b9d-cba7-4d56-8366-874fdc649e8d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.762850 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc5b9d-cba7-4d56-8366-874fdc649e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.768367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 795/1591] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:07.769186 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9f1a2-1dcc-4aa9-879e-8c1a8b104c78 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.780099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9f1a2-1dcc-4aa9-879e-8c1a8b104c78 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.784728 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888cbf4-4094-4509-99d4-661cae70da1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.785314 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888cbf4-4094-4509-99d4-661cae70da1b admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.785594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888cbf4-4094-4509-99d4-661cae70da1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.785867 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888cbf4-4094-4509-99d4-661cae70da1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.786487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888cbf4-4094-4509-99d4-661cae70da1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.788117 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888cbf4-4094-4509-99d4-661cae70da1b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.794115 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-14cc5b9d-cba7-4d56-8366-874fdc649e8d None None] MFA Rules not processed for user `0778bbd644bf4b2ea70a8031ebb12783`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:07.795331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-14cc5b9d-cba7-4d56-8366-874fdc649e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:07.798337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 796/1592] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:07.799195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888cbf4-4094-4509-99d4-661cae70da1b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.809532 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:07.810237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888cbf4-4094-4509-99d4-661cae70da1b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.815493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c8e09-bd2c-4db0-b5be-6ee91acda5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.816244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c8e09-bd2c-4db0-b5be-6ee91acda5d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.816594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c8e09-bd2c-4db0-b5be-6ee91acda5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.817215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c8e09-bd2c-4db0-b5be-6ee91acda5d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.817917 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c8e09-bd2c-4db0-b5be-6ee91acda5d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.819572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c8e09-bd2c-4db0-b5be-6ee91acda5d6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.830560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 797/1593] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:07.831430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c8e09-bd2c-4db0-b5be-6ee91acda5d6 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.841439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c8e09-bd2c-4db0-b5be-6ee91acda5d6 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.846043 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0f85d-17a5-40e0-bc90-28d1b75bfc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.846622 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0f85d-17a5-40e0-bc90-28d1b75bfc7f admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.846901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0f85d-17a5-40e0-bc90-28d1b75bfc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.847175 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0f85d-17a5-40e0-bc90-28d1b75bfc7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.847796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0f85d-17a5-40e0-bc90-28d1b75bfc7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.849409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0f85d-17a5-40e0-bc90-28d1b75bfc7f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.860608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 798/1594] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:07.861378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0f85d-17a5-40e0-bc90-28d1b75bfc7f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.866067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 797/1595] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:07.867030 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc5b9d-cba7-4d56-8366-874fdc649e8d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.871913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0f85d-17a5-40e0-bc90-28d1b75bfc7f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.876561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6e45c-51e1-47b4-ae9b-a50564a6b56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.877236 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6e45c-51e1-47b4-ae9b-a50564a6b56b admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.877518 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6e45c-51e1-47b4-ae9b-a50564a6b56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.877792 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6e45c-51e1-47b4-ae9b-a50564a6b56b admin admin] PATH_INFO: `/v3/projects/688ecc61e9634e738cd742384173461b/users/7a50189fafdc4303893654b48cf6ba4b/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.878592 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc5b9d-cba7-4d56-8366-874fdc649e8d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.883436 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef0899-841e-402d-ad13-9c232a7fce51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.883954 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef0899-841e-402d-ad13-9c232a7fce51 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.884284 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef0899-841e-402d-ad13-9c232a7fce51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.885284 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef0899-841e-402d-ad13-9c232a7fce51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.886157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef0899-841e-402d-ad13-9c232a7fce51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.887765 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef0899-841e-402d-ad13-9c232a7fce51 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.893221 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6e45c-51e1-47b4-ae9b-a50564a6b56b admin admin] RBAC: Authorizing `identity:create_grant(project_id=688ecc61e9634e738cd742384173461b, user_id=7a50189fafdc4303893654b48cf6ba4b, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.895849 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6e45c-51e1-47b4-ae9b-a50564a6b56b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:07.897979 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 798/1596] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:07.899147 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520e8be-142c-427c-ba0f-047172778f6b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.899432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520e8be-142c-427c-ba0f-047172778f6b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.899702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520e8be-142c-427c-ba0f-047172778f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.913172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 799/1597] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:07 2022] PUT /identity/v3/projects/688ecc61e9634e738cd742384173461b/users/7a50189fafdc4303893654b48cf6ba4b/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:07.914049 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6e45c-51e1-47b4-ae9b-a50564a6b56b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:07.942760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c520e8be-142c-427c-ba0f-047172778f6b None None] MFA Rules not processed for user `81d5689e9a344bafb74ca423ffd2b42a`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:07.944000 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c520e8be-142c-427c-ba0f-047172778f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:07.954981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:07.979592 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6e45c-51e1-47b4-ae9b-a50564a6b56b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:07.984155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a2bd3-406c-493b-8841-0716fee271a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:07.984766 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a2bd3-406c-493b-8841-0716fee271a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:07.985036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a2bd3-406c-493b-8841-0716fee271a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:07.985300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a2bd3-406c-493b-8841-0716fee271a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:07.986001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a2bd3-406c-493b-8841-0716fee271a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:07.987594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a2bd3-406c-493b-8841-0716fee271a8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:08.000156 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 800/1598] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:08.001112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a2bd3-406c-493b-8841-0716fee271a8 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.010516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a2bd3-406c-493b-8841-0716fee271a8 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:08.014621 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 799/1599] 10.222.0.26 () {58 vars in 1059 bytes} [Wed Sep 7 07:55:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:08.015460 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520e8be-142c-427c-ba0f-047172778f6b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.016574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc8e51-f627-46aa-8e40-408bdb37b5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:08.017357 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc8e51-f627-46aa-8e40-408bdb37b5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.017642 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc8e51-f627-46aa-8e40-408bdb37b5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.017918 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc8e51-f627-46aa-8e40-408bdb37b5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.018552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc8e51-f627-46aa-8e40-408bdb37b5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:08.020200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc8e51-f627-46aa-8e40-408bdb37b5c4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:08.025999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520e8be-142c-427c-ba0f-047172778f6b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:08.031115 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0228884-fe10-4e49-b963-223fa2fd1244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:08.034830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 801/1600] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:08.034974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0228884-fe10-4e49-b963-223fa2fd1244 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.034974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0228884-fe10-4e49-b963-223fa2fd1244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.034974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0228884-fe10-4e49-b963-223fa2fd1244 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.035256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc8e51-f627-46aa-8e40-408bdb37b5c4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.035353 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0228884-fe10-4e49-b963-223fa2fd1244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:08.035353 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0228884-fe10-4e49-b963-223fa2fd1244 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:08.042809 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc8e51-f627-46aa-8e40-408bdb37b5c4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:08.046339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 800/1601] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:08.047167 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0228884-fe10-4e49-b963-223fa2fd1244 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.049957 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4dc148-139b-41bf-8d73-b2d8d842e343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:08.050648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4dc148-139b-41bf-8d73-b2d8d842e343 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.050913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4dc148-139b-41bf-8d73-b2d8d842e343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.051189 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4dc148-139b-41bf-8d73-b2d8d842e343 admin admin] PATH_INFO: `/v3/projects/b093ac8c5cbc4bb6be326b2fb993a76b/users/4035909c0f414e5ca7fbab2caee29f4d/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.058997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0228884-fe10-4e49-b963-223fa2fd1244 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:08.063642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd0820-18af-4b87-92ee-8a95651a14d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:08.064280 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd0820-18af-4b87-92ee-8a95651a14d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.064564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd0820-18af-4b87-92ee-8a95651a14d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.064837 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd0820-18af-4b87-92ee-8a95651a14d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.065468 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd0820-18af-4b87-92ee-8a95651a14d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:08.067616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd0820-18af-4b87-92ee-8a95651a14d5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:08.068520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4dc148-139b-41bf-8d73-b2d8d842e343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b093ac8c5cbc4bb6be326b2fb993a76b, user_id=4035909c0f414e5ca7fbab2caee29f4d, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:08.071155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4dc148-139b-41bf-8d73-b2d8d842e343 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:08.084802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 802/1602] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:08 2022] PUT /identity/v3/projects/b093ac8c5cbc4bb6be326b2fb993a76b/users/4035909c0f414e5ca7fbab2caee29f4d/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:08.085663 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4dc148-139b-41bf-8d73-b2d8d842e343 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.101251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 801/1603] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:08 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:08.101251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ac890-5503-404a-884b-a982e705c960 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.101251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ac890-5503-404a-884b-a982e705c960 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.101251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ac890-5503-404a-884b-a982e705c960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.128327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-994ac890-5503-404a-884b-a982e705c960 None None] MFA Rules not processed for user `7a50189fafdc4303893654b48cf6ba4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:08.129550 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-994ac890-5503-404a-884b-a982e705c960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:08.149128 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4dc148-139b-41bf-8d73-b2d8d842e343 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:08.171411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:08.199276 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 802/1604] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:08.200185 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ac890-5503-404a-884b-a982e705c960 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.206720 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226252e-ef10-4d60-abf9-8c19cfe1d007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:08.207333 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226252e-ef10-4d60-abf9-8c19cfe1d007 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.207638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226252e-ef10-4d60-abf9-8c19cfe1d007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.207927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226252e-ef10-4d60-abf9-8c19cfe1d007 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.208594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226252e-ef10-4d60-abf9-8c19cfe1d007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:08.210196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226252e-ef10-4d60-abf9-8c19cfe1d007 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:08.213793 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ac890-5503-404a-884b-a982e705c960 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:08.219904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62872bcb-bea5-46fa-b4fd-65550558f887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:08.220527 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62872bcb-bea5-46fa-b4fd-65550558f887 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.221110 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62872bcb-bea5-46fa-b4fd-65550558f887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.221110 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62872bcb-bea5-46fa-b4fd-65550558f887 admin admin] PATH_INFO: `/v3/projects/09a6411dd18c4b6a8fad228058c8e06b/users/e07bc40275734f628fcc2f62dd4dd0e0/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.221883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 803/1605] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:08.222749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226252e-ef10-4d60-abf9-8c19cfe1d007 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.241809 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62872bcb-bea5-46fa-b4fd-65550558f887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09a6411dd18c4b6a8fad228058c8e06b, user_id=e07bc40275734f628fcc2f62dd4dd0e0, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:08.245190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62872bcb-bea5-46fa-b4fd-65550558f887 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:08.258520 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 803/1606] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:08 2022] PUT /identity/v3/projects/09a6411dd18c4b6a8fad228058c8e06b/users/e07bc40275734f628fcc2f62dd4dd0e0/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:08.259448 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62872bcb-bea5-46fa-b4fd-65550558f887 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.271965 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:08.301490 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226252e-ef10-4d60-abf9-8c19cfe1d007 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:08.323801 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62872bcb-bea5-46fa-b4fd-65550558f887 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:08.328495 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd667dfb-9dd3-4b81-82a0-44c277d8f6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:08.329195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd667dfb-9dd3-4b81-82a0-44c277d8f6b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.329473 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd667dfb-9dd3-4b81-82a0-44c277d8f6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.329742 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd667dfb-9dd3-4b81-82a0-44c277d8f6b9 admin admin] PATH_INFO: `/v3/projects/4d3d7139782f4b63ac6dc8d4618abe64/users/b52ececfc19b41b5b8387dc6908d4a4a/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.344610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd667dfb-9dd3-4b81-82a0-44c277d8f6b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d3d7139782f4b63ac6dc8d4618abe64, user_id=b52ececfc19b41b5b8387dc6908d4a4a, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:08.347787 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd667dfb-9dd3-4b81-82a0-44c277d8f6b9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:08.363575 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 804/1607] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:08 2022] PUT /identity/v3/projects/4d3d7139782f4b63ac6dc8d4618abe64/users/b52ececfc19b41b5b8387dc6908d4a4a/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:08.364450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd667dfb-9dd3-4b81-82a0-44c277d8f6b9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.369252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e73ce7-00df-4aa2-9876-28a180a392bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:08.370026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e73ce7-00df-4aa2-9876-28a180a392bb service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.370161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e73ce7-00df-4aa2-9876-28a180a392bb service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.370389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e73ce7-00df-4aa2-9876-28a180a392bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.429022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd667dfb-9dd3-4b81-82a0-44c277d8f6b9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:08.429311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e73ce7-00df-4aa2-9876-28a180a392bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:08.430885 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e73ce7-00df-4aa2-9876-28a180a392bb service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:08.433688 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12a9f9-4f1e-4357-b903-3abdc297472c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:08.434344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12a9f9-4f1e-4357-b903-3abdc297472c admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.434626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12a9f9-4f1e-4357-b903-3abdc297472c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.434951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12a9f9-4f1e-4357-b903-3abdc297472c admin admin] PATH_INFO: `/v3/projects/7a829819301a4a5caa00a85ae0e4fe10/users/f2b563a5c1524b8f9cb3ef9d68eca5be/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.440973 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 804/1608] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:08.442731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce143a3-a5e8-4b55-8b71-afc98c87cb0d None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.443018 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce143a3-a5e8-4b55-8b71-afc98c87cb0d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.443297 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce143a3-a5e8-4b55-8b71-afc98c87cb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.450064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12a9f9-4f1e-4357-b903-3abdc297472c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a829819301a4a5caa00a85ae0e4fe10, user_id=f2b563a5c1524b8f9cb3ef9d68eca5be, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:08.452780 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12a9f9-4f1e-4357-b903-3abdc297472c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:08.463326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1ce143a3-a5e8-4b55-8b71-afc98c87cb0d None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:08.464568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1ce143a3-a5e8-4b55-8b71-afc98c87cb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:08.468153 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 805/1609] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:08 2022] PUT /identity/v3/projects/7a829819301a4a5caa00a85ae0e4fe10/users/f2b563a5c1524b8f9cb3ef9d68eca5be/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:08.469482 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12a9f9-4f1e-4357-b903-3abdc297472c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.500528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:08.530910 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 805/1610] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:08.534469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d8de3-322b-44c8-8128-0d8cc1ab98c6 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.534469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d8de3-322b-44c8-8128-0d8cc1ab98c6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.534469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d8de3-322b-44c8-8128-0d8cc1ab98c6 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.534469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 806/1611] 10.222.0.26 () {54 vars in 962 bytes} [Wed Sep 7 07:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:55:08.535300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d8de3-322b-44c8-8128-0d8cc1ab98c6 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.548157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12a9f9-4f1e-4357-b903-3abdc297472c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:08.552695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d8de3-322b-44c8-8128-0d8cc1ab98c6 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:08.553125 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58518504-93c9-4471-9590-096114335682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:08.553732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58518504-93c9-4471-9590-096114335682 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.554009 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58518504-93c9-4471-9590-096114335682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.554281 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58518504-93c9-4471-9590-096114335682 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.555017 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58518504-93c9-4471-9590-096114335682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:08.557014 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58518504-93c9-4471-9590-096114335682 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:08.557772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31092a2c-bc10-480c-b114-b243a5d4a340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:08.558488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31092a2c-bc10-480c-b114-b243a5d4a340 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.558819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31092a2c-bc10-480c-b114-b243a5d4a340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.559169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31092a2c-bc10-480c-b114-b243a5d4a340 admin admin] PATH_INFO: `/v3/projects/18463a34e4aa4b89a2a9ccad75a0d81e/users/457d69a7306d46cd8445f887eb95c517/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.573368 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:08.578094 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31092a2c-bc10-480c-b114-b243a5d4a340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18463a34e4aa4b89a2a9ccad75a0d81e, user_id=457d69a7306d46cd8445f887eb95c517, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:08.584342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 806/1612] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:08 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:08.584498 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31092a2c-bc10-480c-b114-b243a5d4a340 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:08.584621 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf2c91-eb95-4c43-aef2-2e04676c846c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.584621 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf2c91-eb95-4c43-aef2-2e04676c846c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.584621 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf2c91-eb95-4c43-aef2-2e04676c846c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.598576 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 807/1613] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:08 2022] PUT /identity/v3/projects/18463a34e4aa4b89a2a9ccad75a0d81e/users/457d69a7306d46cd8445f887eb95c517/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:08.600408 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85267fe1-7db8-4e7f-a79e-ec976be0e4f3 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.600884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85267fe1-7db8-4e7f-a79e-ec976be0e4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.601351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85267fe1-7db8-4e7f-a79e-ec976be0e4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.604246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e1cf2c91-eb95-4c43-aef2-2e04676c846c None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:08.605458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e1cf2c91-eb95-4c43-aef2-2e04676c846c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:08.621691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-85267fe1-7db8-4e7f-a79e-ec976be0e4f3 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:08.622870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-85267fe1-7db8-4e7f-a79e-ec976be0e4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:08.688488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 807/1614] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:08.688488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719e467-d5b9-49ac-bb6c-d49ce18d01a2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.688488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719e467-d5b9-49ac-bb6c-d49ce18d01a2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.688488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719e467-d5b9-49ac-bb6c-d49ce18d01a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.703552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 808/1615] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:08.703552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d318098-eaaa-4f62-b800-2a68e7b7d261 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.703552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d318098-eaaa-4f62-b800-2a68e7b7d261 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.703552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d318098-eaaa-4f62-b800-2a68e7b7d261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.728770 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6719e467-d5b9-49ac-bb6c-d49ce18d01a2 None None] MFA Rules not processed for user `f2b563a5c1524b8f9cb3ef9d68eca5be`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:08.730340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6719e467-d5b9-49ac-bb6c-d49ce18d01a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:08.730999 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7d318098-eaaa-4f62-b800-2a68e7b7d261 None None] MFA Rules not processed for user `81d5689e9a344bafb74ca423ffd2b42a`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:08.732197 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7d318098-eaaa-4f62-b800-2a68e7b7d261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:08.794835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 808/1616] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:08.795684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719e467-d5b9-49ac-bb6c-d49ce18d01a2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.798470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 809/1617] 10.222.0.26 () {58 vars in 1059 bytes} [Wed Sep 7 07:55:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:08.805968 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d318098-eaaa-4f62-b800-2a68e7b7d261 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.816676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d318098-eaaa-4f62-b800-2a68e7b7d261 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:08.821464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc3ca8-8357-4da7-9332-8ec683979d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:08.822107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc3ca8-8357-4da7-9332-8ec683979d80 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.822422 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc3ca8-8357-4da7-9332-8ec683979d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.822638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc3ca8-8357-4da7-9332-8ec683979d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.823264 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc3ca8-8357-4da7-9332-8ec683979d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:08.824898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc3ca8-8357-4da7-9332-8ec683979d80 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:08.835240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 810/1618] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:08.836692 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f6f9f-daab-4e79-bc48-7746369adfe5 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.837155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f6f9f-daab-4e79-bc48-7746369adfe5 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.837457 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f6f9f-daab-4e79-bc48-7746369adfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.842488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:08.855262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719e467-d5b9-49ac-bb6c-d49ce18d01a2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:08.856525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-eb6f6f9f-daab-4e79-bc48-7746369adfe5 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:08.857721 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-eb6f6f9f-daab-4e79-bc48-7746369adfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:08.859159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa99528-9f37-4dc1-9080-6d7ba895bd66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:08.859721 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa99528-9f37-4dc1-9080-6d7ba895bd66 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.859996 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa99528-9f37-4dc1-9080-6d7ba895bd66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.860289 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa99528-9f37-4dc1-9080-6d7ba895bd66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.865465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 811/1619] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:08.866038 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f6f9f-daab-4e79-bc48-7746369adfe5 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.873077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa99528-9f37-4dc1-9080-6d7ba895bd66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:08.874530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa99528-9f37-4dc1-9080-6d7ba895bd66 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:08.875105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f6f9f-daab-4e79-bc48-7746369adfe5 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:08.878653 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f1469-69ed-44fb-a348-3ee5267ea144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:08.879246 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f1469-69ed-44fb-a348-3ee5267ea144 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.879523 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f1469-69ed-44fb-a348-3ee5267ea144 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.879795 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f1469-69ed-44fb-a348-3ee5267ea144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.885761 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 809/1620] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:08.886572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa99528-9f37-4dc1-9080-6d7ba895bd66 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.889864 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f1469-69ed-44fb-a348-3ee5267ea144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:08.891301 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f1469-69ed-44fb-a348-3ee5267ea144 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:08.894968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa99528-9f37-4dc1-9080-6d7ba895bd66 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:08.899282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e9b72-1f48-4992-8d6e-91128b4760cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:08.899866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e9b72-1f48-4992-8d6e-91128b4760cb service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.900238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e9b72-1f48-4992-8d6e-91128b4760cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.900705 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e9b72-1f48-4992-8d6e-91128b4760cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.901961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 812/1621] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:08.903337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdebd2-e14a-4c67-8696-d5c6e7146b50 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.903629 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdebd2-e14a-4c67-8696-d5c6e7146b50 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.903901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdebd2-e14a-4c67-8696-d5c6e7146b50 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.905425 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 813/1622] 10.222.0.26 () {54 vars in 912 bytes} [Wed Sep 7 07:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:55:08.906130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdebd2-e14a-4c67-8696-d5c6e7146b50 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.910823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e9b72-1f48-4992-8d6e-91128b4760cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:08.913336 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e9b72-1f48-4992-8d6e-91128b4760cb service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:08.923952 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdebd2-e14a-4c67-8696-d5c6e7146b50 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:08.924321 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 810/1623] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:08.925139 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e9b72-1f48-4992-8d6e-91128b4760cb service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:08.932710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2c7c2-a768-4f86-8630-accfe65febfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:08.933200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2c7c2-a768-4f86-8630-accfe65febfe service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:08.933508 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2c7c2-a768-4f86-8630-accfe65febfe service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:08.933746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2c7c2-a768-4f86-8630-accfe65febfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:08.996212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e9b72-1f48-4992-8d6e-91128b4760cb service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:09.000953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fff58-27f3-4a8d-9113-171d19142709 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:09.001762 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fff58-27f3-4a8d-9113-171d19142709 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:09.002063 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fff58-27f3-4a8d-9113-171d19142709 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:09.002476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fff58-27f3-4a8d-9113-171d19142709 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:09.014552 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fff58-27f3-4a8d-9113-171d19142709 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:09.014947 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:09.016630 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fff58-27f3-4a8d-9113-171d19142709 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:09.028800 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2c7c2-a768-4f86-8630-accfe65febfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:09.030461 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 811/1624] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:09.030718 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2c7c2-a768-4f86-8630-accfe65febfe service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:09.042381 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 814/1625] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:09.044119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fff58-27f3-4a8d-9113-171d19142709 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:09.054890 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2c7c2-a768-4f86-8630-accfe65febfe service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:09.066597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fff58-27f3-4a8d-9113-171d19142709 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:09.081462 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15997b8d-eeeb-4797-a1ca-56a60275bd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:09.081622 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2c7c2-a768-4f86-8630-accfe65febfe service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:09.081712 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15997b8d-eeeb-4797-a1ca-56a60275bd61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:09.081712 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15997b8d-eeeb-4797-a1ca-56a60275bd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:09.082000 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15997b8d-eeeb-4797-a1ca-56a60275bd61 admin admin] PATH_INFO: `/v3/projects/c1fd5524ee8f4fa19d6f8dcb4bad3eed/users/2a7297b50b834fc29399341518bb22d2/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:09.099692 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bde55-f076-43ac-811e-54c0991a50ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:09.100388 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bde55-f076-43ac-811e-54c0991a50ff service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:09.100666 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bde55-f076-43ac-811e-54c0991a50ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:09.100933 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bde55-f076-43ac-811e-54c0991a50ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:09.113673 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15997b8d-eeeb-4797-a1ca-56a60275bd61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1fd5524ee8f4fa19d6f8dcb4bad3eed, user_id=2a7297b50b834fc29399341518bb22d2, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:09.117696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15997b8d-eeeb-4797-a1ca-56a60275bd61 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:09.129677 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bde55-f076-43ac-811e-54c0991a50ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:09.133221 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bde55-f076-43ac-811e-54c0991a50ff service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:09.135497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 812/1626] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:09 2022] PUT /identity/v3/projects/c1fd5524ee8f4fa19d6f8dcb4bad3eed/users/2a7297b50b834fc29399341518bb22d2/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:09.137237 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c793f65-650e-4bc6-b99b-5e6627b6ccd1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:09.137517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c793f65-650e-4bc6-b99b-5e6627b6ccd1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:09.137793 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c793f65-650e-4bc6-b99b-5e6627b6ccd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:09.155726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 815/1627] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:09.156620 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bde55-f076-43ac-811e-54c0991a50ff service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:09.166821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bde55-f076-43ac-811e-54c0991a50ff service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:09.171358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8c793f65-650e-4bc6-b99b-5e6627b6ccd1 None None] MFA Rules not processed for user `81d5689e9a344bafb74ca423ffd2b42a`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:09.174408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8c793f65-650e-4bc6-b99b-5e6627b6ccd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:09.238892 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4760b-e770-4709-9376-9107a93d83a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:09.239539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4760b-e770-4709-9376-9107a93d83a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:09.239772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4760b-e770-4709-9376-9107a93d83a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:09.240210 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4760b-e770-4709-9376-9107a93d83a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:09.240926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4760b-e770-4709-9376-9107a93d83a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:09.245970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4760b-e770-4709-9376-9107a93d83a0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:09.260736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 816/1628] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:09.261635 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4760b-e770-4709-9376-9107a93d83a0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:09.275905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4760b-e770-4709-9376-9107a93d83a0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:09.283069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2b8fa-5721-4419-b370-b047a9c2d7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:09.283941 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2b8fa-5721-4419-b370-b047a9c2d7eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:09.284280 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2b8fa-5721-4419-b370-b047a9c2d7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:09.284555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2b8fa-5721-4419-b370-b047a9c2d7eb admin admin] PATH_INFO: `/v3/projects/c1fd5524ee8f4fa19d6f8dcb4bad3eed/users/2a7297b50b834fc29399341518bb22d2/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:09.287235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2b8fa-5721-4419-b370-b047a9c2d7eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1fd5524ee8f4fa19d6f8dcb4bad3eed, user_id=2a7297b50b834fc29399341518bb22d2, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:09.289935 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2b8fa-5721-4419-b370-b047a9c2d7eb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:09.305540 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 817/1629] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:09 2022] PUT /identity/v3/projects/c1fd5524ee8f4fa19d6f8dcb4bad3eed/users/2a7297b50b834fc29399341518bb22d2/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:09.317817 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2dcc8-6257-48b5-8ee1-99a587af7b41 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:09.318123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2dcc8-6257-48b5-8ee1-99a587af7b41 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:09.318372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2dcc8-6257-48b5-8ee1-99a587af7b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:09.342584 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 813/1630] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Sep 7 07:55:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:09.358449 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b5b2dcc8-6257-48b5-8ee1-99a587af7b41 None None] MFA Rules not processed for user `2a7297b50b834fc29399341518bb22d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:09.359675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b5b2dcc8-6257-48b5-8ee1-99a587af7b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:09.360470 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c793f65-650e-4bc6-b99b-5e6627b6ccd1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:09.457015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 818/1631] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:09.475251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:09.497342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c793f65-650e-4bc6-b99b-5e6627b6ccd1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:09.606123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b25ba4-823f-4adf-bc90-60722f613177 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:09.606937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b25ba4-823f-4adf-bc90-60722f613177 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:09.607285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b25ba4-823f-4adf-bc90-60722f613177 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:09.608991 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b25ba4-823f-4adf-bc90-60722f613177 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:09.613279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff55cd2-3344-48ba-9280-10c1451eec3f None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:09.613958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff55cd2-3344-48ba-9280-10c1451eec3f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:09.614214 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff55cd2-3344-48ba-9280-10c1451eec3f None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:09.616099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 819/1632] 10.222.0.26 () {54 vars in 912 bytes} [Wed Sep 7 07:55:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:55:09.631897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a8f77-a802-4aa3-b688-43c3df073ec3 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:09.632554 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a8f77-a802-4aa3-b688-43c3df073ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:09.632830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a8f77-a802-4aa3-b688-43c3df073ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:09.671192 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-805a8f77-a802-4aa3-b688-43c3df073ec3 None None] MFA Rules not processed for user `81d5689e9a344bafb74ca423ffd2b42a`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:09.672565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-805a8f77-a802-4aa3-b688-43c3df073ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:09.721912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b25ba4-823f-4adf-bc90-60722f613177 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:09.724656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b25ba4-823f-4adf-bc90-60722f613177 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:09.727671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:09.745239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 814/1633] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:09.759897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 820/1634] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Sep 7 07:55:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:09.836733 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b25ba4-823f-4adf-bc90-60722f613177 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:09.856537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b25ba4-823f-4adf-bc90-60722f613177 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:09.866531 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3592ac-ebe0-4a27-9d9a-330a8f7b8936 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:09.868468 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3592ac-ebe0-4a27-9d9a-330a8f7b8936 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:09.868821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3592ac-ebe0-4a27-9d9a-330a8f7b8936 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:09.871273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3592ac-ebe0-4a27-9d9a-330a8f7b8936 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:09.907830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3592ac-ebe0-4a27-9d9a-330a8f7b8936 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:09.909235 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3592ac-ebe0-4a27-9d9a-330a8f7b8936 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:09.939957 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 815/1635] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:09.987498 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a8f77-a802-4aa3-b688-43c3df073ec3 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:09.998393 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a8f77-a802-4aa3-b688-43c3df073ec3 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:10.016178 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebfbfb-03b9-41b6-ab1a-fe9f351dde4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:10.017262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebfbfb-03b9-41b6-ab1a-fe9f351dde4d service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:10.017677 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebfbfb-03b9-41b6-ab1a-fe9f351dde4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:10.018096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebfbfb-03b9-41b6-ab1a-fe9f351dde4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:10.043074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebfbfb-03b9-41b6-ab1a-fe9f351dde4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:10.044992 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebfbfb-03b9-41b6-ab1a-fe9f351dde4d service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:10.111741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 821/1636] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:10.348174 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3592ac-ebe0-4a27-9d9a-330a8f7b8936 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:10.362312 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3592ac-ebe0-4a27-9d9a-330a8f7b8936 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:10.362768 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebfbfb-03b9-41b6-ab1a-fe9f351dde4d service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:10.371854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b7ec2-0cb3-4f8b-a5e9-3b112d9e8d47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:10.372495 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b7ec2-0cb3-4f8b-a5e9-3b112d9e8d47 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:10.372767 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b7ec2-0cb3-4f8b-a5e9-3b112d9e8d47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:10.373035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b7ec2-0cb3-4f8b-a5e9-3b112d9e8d47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:10.513036 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b7ec2-0cb3-4f8b-a5e9-3b112d9e8d47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:10.514516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b7ec2-0cb3-4f8b-a5e9-3b112d9e8d47 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:10.532526 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 816/1637] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:10.571216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebfbfb-03b9-41b6-ab1a-fe9f351dde4d service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:10.582595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688ae17-e14b-4254-a1b6-42417bf22e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:10.588656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688ae17-e14b-4254-a1b6-42417bf22e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:10.589161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688ae17-e14b-4254-a1b6-42417bf22e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:10.589504 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688ae17-e14b-4254-a1b6-42417bf22e98 admin admin] PATH_INFO: `/v3/users/04e229ddb87d4e6eaff2fd433ec2606c` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:10.594107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688ae17-e14b-4254-a1b6-42417bf22e98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04e229ddb87d4e6eaff2fd433ec2606c)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:10.597512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: WARNING py.warnings [None req-9688ae17-e14b-4254-a1b6-42417bf22e98 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 07:55:10.597512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: warnings.warn(msg) Sep 07 07:55:10.597512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: Sep 07 07:55:10.605399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688ae17-e14b-4254-a1b6-42417bf22e98 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:10.761679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9688ae17-e14b-4254-a1b6-42417bf22e98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04e229ddb87d4e6eaff2fd433ec2606c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:10.763618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9688ae17-e14b-4254-a1b6-42417bf22e98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04e229ddb87d4e6eaff2fd433ec2606c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:10.834923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9688ae17-e14b-4254-a1b6-42417bf22e98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04e229ddb87d4e6eaff2fd433ec2606c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:10.838990 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9688ae17-e14b-4254-a1b6-42417bf22e98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04e229ddb87d4e6eaff2fd433ec2606c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:10.891397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9688ae17-e14b-4254-a1b6-42417bf22e98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04e229ddb87d4e6eaff2fd433ec2606c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:10.939618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 822/1638] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:10 2022] DELETE /identity/v3/users/04e229ddb87d4e6eaff2fd433ec2606c => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:10.946415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e485c-d35a-4008-a0c5-f8c90ae84ee1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:10.948123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e485c-d35a-4008-a0c5-f8c90ae84ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:10.948838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e485c-d35a-4008-a0c5-f8c90ae84ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:10.981170 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6e3e485c-d35a-4008-a0c5-f8c90ae84ee1 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:10.982211 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6e3e485c-d35a-4008-a0c5-f8c90ae84ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:11.022207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:11.154517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 817/1639] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:55:11.171973 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688ae17-e14b-4254-a1b6-42417bf22e98 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:11.173629 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9688ae17-e14b-4254-a1b6-42417bf22e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:11.298672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:11.325409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688ae17-e14b-4254-a1b6-42417bf22e98 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:11.330319 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52af51c-d696-445b-b0cb-2ea7a6f99111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:11.331164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52af51c-d696-445b-b0cb-2ea7a6f99111 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:11.331444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52af51c-d696-445b-b0cb-2ea7a6f99111 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:11.331715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52af51c-d696-445b-b0cb-2ea7a6f99111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:11.349296 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52af51c-d696-445b-b0cb-2ea7a6f99111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:11.351265 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52af51c-d696-445b-b0cb-2ea7a6f99111 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:11.373834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 823/1640] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:14.074025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e485c-d35a-4008-a0c5-f8c90ae84ee1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:14.075471 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6e3e485c-d35a-4008-a0c5-f8c90ae84ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:14.097650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e485c-d35a-4008-a0c5-f8c90ae84ee1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:14.104214 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad6ab0-222d-4ad8-9c96-b703b5d36587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:14.105058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad6ab0-222d-4ad8-9c96-b703b5d36587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:14.106107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad6ab0-222d-4ad8-9c96-b703b5d36587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:14.106499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad6ab0-222d-4ad8-9c96-b703b5d36587 admin admin] PATH_INFO: `/v3/projects/94ef8ab5782f4d4cb8ceb6d8b78911ba` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:14.109889 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad6ab0-222d-4ad8-9c96-b703b5d36587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94ef8ab5782f4d4cb8ceb6d8b78911ba)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:14.111043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-88ad6ab0-222d-4ad8-9c96-b703b5d36587 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 07:55:14.111043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Sep 07 07:55:14.111043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: Sep 07 07:55:14.114440 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad6ab0-222d-4ad8-9c96-b703b5d36587 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:14.247584 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-88ad6ab0-222d-4ad8-9c96-b703b5d36587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94ef8ab5782f4d4cb8ceb6d8b78911ba', 'request_id': 'req-88ad6ab0-222d-4ad8-9c96-b703b5d36587'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:14.261821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-88ad6ab0-222d-4ad8-9c96-b703b5d36587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94ef8ab5782f4d4cb8ceb6d8b78911ba', 'request_id': 'req-88ad6ab0-222d-4ad8-9c96-b703b5d36587'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:14.297288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-88ad6ab0-222d-4ad8-9c96-b703b5d36587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94ef8ab5782f4d4cb8ceb6d8b78911ba', 'request_id': 'req-88ad6ab0-222d-4ad8-9c96-b703b5d36587'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:14.320234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52af51c-d696-445b-b0cb-2ea7a6f99111 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:14.325514 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d52af51c-d696-445b-b0cb-2ea7a6f99111 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:14.329682 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-88ad6ab0-222d-4ad8-9c96-b703b5d36587 admin admin] The token cache is being invalidate because project 94ef8ab5782f4d4cb8ceb6d8b78911ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:14.330130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-88ad6ab0-222d-4ad8-9c96-b703b5d36587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:14.331278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-88ad6ab0-222d-4ad8-9c96-b703b5d36587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:14.333143 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 818/1641] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:14 2022] DELETE /identity/v3/projects/94ef8ab5782f4d4cb8ceb6d8b78911ba => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:14.340016 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb47f41-0231-4f66-bf4e-18a67c808f5b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:14.340446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb47f41-0231-4f66-bf4e-18a67c808f5b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:14.340818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb47f41-0231-4f66-bf4e-18a67c808f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:14.366906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-eeb47f41-0231-4f66-bf4e-18a67c808f5b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:14.367511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52af51c-d696-445b-b0cb-2ea7a6f99111 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:14.368996 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-eeb47f41-0231-4f66-bf4e-18a67c808f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:14.451028 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 819/1642] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:14.458620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e89eb5-f51c-4a6e-8598-f241e0bafaaa None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:14.459088 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e89eb5-f51c-4a6e-8598-f241e0bafaaa None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:14.459464 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e89eb5-f51c-4a6e-8598-f241e0bafaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:14.516806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d4e89eb5-f51c-4a6e-8598-f241e0bafaaa None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:14.522079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d4e89eb5-f51c-4a6e-8598-f241e0bafaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:14.546252 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 820/1643] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:55:14.551289 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e89eb5-f51c-4a6e-8598-f241e0bafaaa None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:14.573381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e89eb5-f51c-4a6e-8598-f241e0bafaaa None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:14.576282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281a108-faae-4195-847d-61bea0aada31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:14.577134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281a108-faae-4195-847d-61bea0aada31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:14.577562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281a108-faae-4195-847d-61bea0aada31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:14.577967 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281a108-faae-4195-847d-61bea0aada31 admin admin] PATH_INFO: `/v3/users/7a50189fafdc4303893654b48cf6ba4b` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:14.580127 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281a108-faae-4195-847d-61bea0aada31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a50189fafdc4303893654b48cf6ba4b)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:14.582423 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281a108-faae-4195-847d-61bea0aada31 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:14.591200 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d113b-65bd-4ef0-8289-df5d9f485b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:14.592907 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d113b-65bd-4ef0-8289-df5d9f485b02 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:14.595975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d113b-65bd-4ef0-8289-df5d9f485b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:14.596398 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d113b-65bd-4ef0-8289-df5d9f485b02 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:14.601214 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d113b-65bd-4ef0-8289-df5d9f485b02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:14.602774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d113b-65bd-4ef0-8289-df5d9f485b02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:14.610968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d113b-65bd-4ef0-8289-df5d9f485b02 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:14.647591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 821/1644] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:55:14.652761 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d113b-65bd-4ef0-8289-df5d9f485b02 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:14.683695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d113b-65bd-4ef0-8289-df5d9f485b02 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:14.694483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4281a108-faae-4195-847d-61bea0aada31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a50189fafdc4303893654b48cf6ba4b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:14.695187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4281a108-faae-4195-847d-61bea0aada31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a50189fafdc4303893654b48cf6ba4b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:14.697633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fffbf-4772-4594-97b1-e2c3a8c64379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:14.701075 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fffbf-4772-4594-97b1-e2c3a8c64379 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:14.701457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fffbf-4772-4594-97b1-e2c3a8c64379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:14.701869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fffbf-4772-4594-97b1-e2c3a8c64379 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:14.704218 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1fffbf-4772-4594-97b1-e2c3a8c64379 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:14.708665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1fffbf-4772-4594-97b1-e2c3a8c64379 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:14.725242 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4281a108-faae-4195-847d-61bea0aada31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a50189fafdc4303893654b48cf6ba4b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:14.726024 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4281a108-faae-4195-847d-61bea0aada31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a50189fafdc4303893654b48cf6ba4b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:14.741279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4281a108-faae-4195-847d-61bea0aada31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a50189fafdc4303893654b48cf6ba4b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:14.768242 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 824/1645] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:14 2022] DELETE /identity/v3/users/7a50189fafdc4303893654b48cf6ba4b => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:14.774664 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d930c4-3c2a-486b-a3fb-836a623aba99 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:14.778958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d930c4-3c2a-486b-a3fb-836a623aba99 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:14.780035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d930c4-3c2a-486b-a3fb-836a623aba99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:14.787216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 822/1646] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:14 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:14.792981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fffbf-4772-4594-97b1-e2c3a8c64379 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:14.797291 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4d1fffbf-4772-4594-97b1-e2c3a8c64379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:14.812738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:14.851744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b1d930c4-3c2a-486b-a3fb-836a623aba99 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:14.858139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b1d930c4-3c2a-486b-a3fb-836a623aba99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:14.891745 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fffbf-4772-4594-97b1-e2c3a8c64379 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:14.899120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51acdeec-5d87-4658-bc32-0824939fc351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:14.900776 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acdeec-5d87-4658-bc32-0824939fc351 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:14.901154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acdeec-5d87-4658-bc32-0824939fc351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:14.901712 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acdeec-5d87-4658-bc32-0824939fc351 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:14.902464 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51acdeec-5d87-4658-bc32-0824939fc351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:14.905096 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51acdeec-5d87-4658-bc32-0824939fc351 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:14.942297 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 823/1647] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:14 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:14.944532 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51acdeec-5d87-4658-bc32-0824939fc351 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:14.971713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51acdeec-5d87-4658-bc32-0824939fc351 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:14.988413 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a27827-e034-4f41-aa9a-c042197554a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:14.989138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a27827-e034-4f41-aa9a-c042197554a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:14.992347 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a27827-e034-4f41-aa9a-c042197554a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:14.994828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a27827-e034-4f41-aa9a-c042197554a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:14.998268 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a27827-e034-4f41-aa9a-c042197554a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:15.018440 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a27827-e034-4f41-aa9a-c042197554a5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:15.059333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 824/1648] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:15.060460 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a27827-e034-4f41-aa9a-c042197554a5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:15.075079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 825/1649] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:15.085891 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a27827-e034-4f41-aa9a-c042197554a5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:15.100571 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d930c4-3c2a-486b-a3fb-836a623aba99 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:15.100948 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4511e-8e54-4563-b865-7319be0f1f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:15.101627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4511e-8e54-4563-b865-7319be0f1f1c admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:15.102078 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b1d930c4-3c2a-486b-a3fb-836a623aba99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:15.103537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4511e-8e54-4563-b865-7319be0f1f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:15.103911 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4511e-8e54-4563-b865-7319be0f1f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:15.104508 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4511e-8e54-4563-b865-7319be0f1f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:15.107301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4511e-8e54-4563-b865-7319be0f1f1c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:15.116898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:15.142740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 825/1650] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:15.143649 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4511e-8e54-4563-b865-7319be0f1f1c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:15.167211 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4511e-8e54-4563-b865-7319be0f1f1c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:15.178011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f434d-4b4a-4293-a568-6f4f0d53ea92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:15.178693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f434d-4b4a-4293-a568-6f4f0d53ea92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:15.178974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f434d-4b4a-4293-a568-6f4f0d53ea92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:15.179247 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f434d-4b4a-4293-a568-6f4f0d53ea92 admin admin] PATH_INFO: `/v3/projects/045df47d901b4fb3a0f7ad9cc8e0f9c2/users/d13d5bb983d749cdbef7dcb2a18b0fd5/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:15.214666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f434d-4b4a-4293-a568-6f4f0d53ea92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=045df47d901b4fb3a0f7ad9cc8e0f9c2, user_id=d13d5bb983d749cdbef7dcb2a18b0fd5, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:15.217559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f434d-4b4a-4293-a568-6f4f0d53ea92 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:15.222840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d930c4-3c2a-486b-a3fb-836a623aba99 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:15.233545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69946e34-1721-48aa-a239-9a90392ad7ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:15.234184 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69946e34-1721-48aa-a239-9a90392ad7ce service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:15.234453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69946e34-1721-48aa-a239-9a90392ad7ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:15.234728 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69946e34-1721-48aa-a239-9a90392ad7ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:15.245942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 826/1651] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:15 2022] PUT /identity/v3/projects/045df47d901b4fb3a0f7ad9cc8e0f9c2/users/d13d5bb983d749cdbef7dcb2a18b0fd5/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:15.265530 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69946e34-1721-48aa-a239-9a90392ad7ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:15.265832 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c3882-19a6-4a14-9d57-3f6192ea4d3b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:15.265832 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c3882-19a6-4a14-9d57-3f6192ea4d3b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:15.265832 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c3882-19a6-4a14-9d57-3f6192ea4d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:15.266010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69946e34-1721-48aa-a239-9a90392ad7ce service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:15.283129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 826/1652] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:15.307313 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f98c3882-19a6-4a14-9d57-3f6192ea4d3b None None] MFA Rules not processed for user `d13d5bb983d749cdbef7dcb2a18b0fd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:15.308859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f98c3882-19a6-4a14-9d57-3f6192ea4d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:15.447696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 827/1653] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:15.451279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69946e34-1721-48aa-a239-9a90392ad7ce service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:15.464298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69946e34-1721-48aa-a239-9a90392ad7ce service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:15.605456 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3edd4-504f-4026-9b8e-f0e175d53ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:15.606084 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3edd4-504f-4026-9b8e-f0e175d53ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:15.606360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3edd4-504f-4026-9b8e-f0e175d53ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:15.606621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3edd4-504f-4026-9b8e-f0e175d53ef2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:15.610126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3edd4-504f-4026-9b8e-f0e175d53ef2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:15.610126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3edd4-504f-4026-9b8e-f0e175d53ef2 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:15.672567 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 827/1654] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:15 2022] POST /identity/v3/projects => generated 369 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:15.676313 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c3882-19a6-4a14-9d57-3f6192ea4d3b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:15.697715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c3882-19a6-4a14-9d57-3f6192ea4d3b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:15.834408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:15.893947 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc1e70-b8d6-4001-b0c0-6c321acb4fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:15.894592 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc1e70-b8d6-4001-b0c0-6c321acb4fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:15.894870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc1e70-b8d6-4001-b0c0-6c321acb4fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:15.895141 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc1e70-b8d6-4001-b0c0-6c321acb4fc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:15.895783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafc1e70-b8d6-4001-b0c0-6c321acb4fc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:15.897748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafc1e70-b8d6-4001-b0c0-6c321acb4fc7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:15.986054 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 828/1655] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:15 2022] POST /identity/v3/users => generated 291 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:15.989671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3edd4-504f-4026-9b8e-f0e175d53ef2 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:16.006404 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3edd4-504f-4026-9b8e-f0e175d53ef2 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:16.019984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc129231-ed35-466c-a36d-50f2af5065a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:16.021009 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc129231-ed35-466c-a36d-50f2af5065a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:16.021485 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc129231-ed35-466c-a36d-50f2af5065a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:16.021952 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc129231-ed35-466c-a36d-50f2af5065a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:16.022813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc129231-ed35-466c-a36d-50f2af5065a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:16.025200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc129231-ed35-466c-a36d-50f2af5065a8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:16.051989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 828/1656] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:16.057364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc1e70-b8d6-4001-b0c0-6c321acb4fc7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:16.081730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc1e70-b8d6-4001-b0c0-6c321acb4fc7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:16.100766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bce3bd-2ae6-4065-b940-58a8cbfab5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:16.102404 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bce3bd-2ae6-4065-b940-58a8cbfab5fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:16.103444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bce3bd-2ae6-4065-b940-58a8cbfab5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:16.103873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bce3bd-2ae6-4065-b940-58a8cbfab5fb admin admin] PATH_INFO: `/v3/projects/eb00ba984ba64798893686720da1e14e/users/e07e00059684444794bf17a766fe16f0/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:16.141278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bce3bd-2ae6-4065-b940-58a8cbfab5fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb00ba984ba64798893686720da1e14e, user_id=e07e00059684444794bf17a766fe16f0, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:16.144398 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bce3bd-2ae6-4065-b940-58a8cbfab5fb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:16.176537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 829/1657] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:16 2022] PUT /identity/v3/projects/eb00ba984ba64798893686720da1e14e/users/e07e00059684444794bf17a766fe16f0/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:16.178039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc129231-ed35-466c-a36d-50f2af5065a8 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:16.195346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc129231-ed35-466c-a36d-50f2af5065a8 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:16.355862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7b05e-f98a-4ea9-80b5-a9db0d76c78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:16.356630 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7b05e-f98a-4ea9-80b5-a9db0d76c78d admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:16.357031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7b05e-f98a-4ea9-80b5-a9db0d76c78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:16.357823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7b05e-f98a-4ea9-80b5-a9db0d76c78d admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:16.358146 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7b05e-f98a-4ea9-80b5-a9db0d76c78d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:16.359855 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7b05e-f98a-4ea9-80b5-a9db0d76c78d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:16.394488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 829/1658] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:16.398745 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bce3bd-2ae6-4065-b940-58a8cbfab5fb admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:16.412879 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bce3bd-2ae6-4065-b940-58a8cbfab5fb admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:16.463699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1311fd9-0405-4594-88cc-e1e1180312f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:16.464508 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1311fd9-0405-4594-88cc-e1e1180312f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:16.464861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1311fd9-0405-4594-88cc-e1e1180312f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:16.465199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1311fd9-0405-4594-88cc-e1e1180312f5 admin admin] PATH_INFO: `/v3/projects/eb00ba984ba64798893686720da1e14e/users/e07e00059684444794bf17a766fe16f0/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:16.470882 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1311fd9-0405-4594-88cc-e1e1180312f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb00ba984ba64798893686720da1e14e, user_id=e07e00059684444794bf17a766fe16f0, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:16.473635 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1311fd9-0405-4594-88cc-e1e1180312f5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:16.513693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 830/1659] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:16 2022] PUT /identity/v3/projects/eb00ba984ba64798893686720da1e14e/users/e07e00059684444794bf17a766fe16f0/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:16.523848 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d61f84-db72-4d36-b897-07cf210a42d7 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:16.524975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d61f84-db72-4d36-b897-07cf210a42d7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:16.525577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d61f84-db72-4d36-b897-07cf210a42d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:16.585223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e0d61f84-db72-4d36-b897-07cf210a42d7 None None] MFA Rules not processed for user `e07e00059684444794bf17a766fe16f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:16.585223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e0d61f84-db72-4d36-b897-07cf210a42d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:16.623980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:16.809808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 830/1660] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:16.823180 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1311fd9-0405-4594-88cc-e1e1180312f5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:16.825038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c1311fd9-0405-4594-88cc-e1e1180312f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:16.983506 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1311fd9-0405-4594-88cc-e1e1180312f5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:16.994428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84e581-1066-4455-b229-48193b7943f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:16.999612 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84e581-1066-4455-b229-48193b7943f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:17.000166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84e581-1066-4455-b229-48193b7943f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:17.001191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84e581-1066-4455-b229-48193b7943f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:17.036370 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84e581-1066-4455-b229-48193b7943f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:17.044535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84e581-1066-4455-b229-48193b7943f0 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:17.062042 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 831/1661] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:17.223872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d61f84-db72-4d36-b897-07cf210a42d7 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:17.239292 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d61f84-db72-4d36-b897-07cf210a42d7 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:17.243844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca146e90-ade3-4eda-82d6-6ea99e82fd47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:17.244585 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca146e90-ade3-4eda-82d6-6ea99e82fd47 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:17.244961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca146e90-ade3-4eda-82d6-6ea99e82fd47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:17.245324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca146e90-ade3-4eda-82d6-6ea99e82fd47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:17.417564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca146e90-ade3-4eda-82d6-6ea99e82fd47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:17.421415 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca146e90-ade3-4eda-82d6-6ea99e82fd47 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:17.467754 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 831/1662] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:17.880233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84e581-1066-4455-b229-48193b7943f0 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:17.882976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ae84e581-1066-4455-b229-48193b7943f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:18.036940 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84e581-1066-4455-b229-48193b7943f0 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:18.043001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85479d-f6e0-4601-b2a0-dd063ba5a19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:18.044020 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85479d-f6e0-4601-b2a0-dd063ba5a19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:18.044528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85479d-f6e0-4601-b2a0-dd063ba5a19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:18.045031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85479d-f6e0-4601-b2a0-dd063ba5a19a admin admin] PATH_INFO: `/v3/projects/688ecc61e9634e738cd742384173461b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:18.047496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85479d-f6e0-4601-b2a0-dd063ba5a19a admin admin] RBAC: Authorizing `identity:delete_project(project_id=688ecc61e9634e738cd742384173461b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:18.050410 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85479d-f6e0-4601-b2a0-dd063ba5a19a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:18.125533 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:18.149961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1e85479d-f6e0-4601-b2a0-dd063ba5a19a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '688ecc61e9634e738cd742384173461b', 'request_id': 'req-1e85479d-f6e0-4601-b2a0-dd063ba5a19a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:18.177644 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1e85479d-f6e0-4601-b2a0-dd063ba5a19a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '688ecc61e9634e738cd742384173461b', 'request_id': 'req-1e85479d-f6e0-4601-b2a0-dd063ba5a19a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:18.185415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1e85479d-f6e0-4601-b2a0-dd063ba5a19a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '688ecc61e9634e738cd742384173461b', 'request_id': 'req-1e85479d-f6e0-4601-b2a0-dd063ba5a19a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:18.214743 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1e85479d-f6e0-4601-b2a0-dd063ba5a19a admin admin] The token cache is being invalidate because project 688ecc61e9634e738cd742384173461b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:18.215060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1e85479d-f6e0-4601-b2a0-dd063ba5a19a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:18.215931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1e85479d-f6e0-4601-b2a0-dd063ba5a19a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:18.218705 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 832/1663] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:17 2022] DELETE /identity/v3/projects/688ecc61e9634e738cd742384173461b => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:18.224642 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a3f4d-ce8f-48fa-9863-f0d7814fc625 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:18.225865 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a3f4d-ce8f-48fa-9863-f0d7814fc625 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:18.226478 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a3f4d-ce8f-48fa-9863-f0d7814fc625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:18.265539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-754a3f4d-ce8f-48fa-9863-f0d7814fc625 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:18.283318 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-754a3f4d-ce8f-48fa-9863-f0d7814fc625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:18.542025 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 832/1664] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:18.555253 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf94ef-9247-41be-ab6d-068270d662a2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:18.555545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf94ef-9247-41be-ab6d-068270d662a2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:18.555885 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf94ef-9247-41be-ab6d-068270d662a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:18.610619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3dcf94ef-9247-41be-ab6d-068270d662a2 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:18.611874 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3dcf94ef-9247-41be-ab6d-068270d662a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:18.646855 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 833/1665] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:18.651901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a3f4d-ce8f-48fa-9863-f0d7814fc625 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:18.668039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a3f4d-ce8f-48fa-9863-f0d7814fc625 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:18.683729 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927a8fa-635e-4447-8f40-571537bca3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:18.684386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927a8fa-635e-4447-8f40-571537bca3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:18.684661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927a8fa-635e-4447-8f40-571537bca3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:18.684935 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927a8fa-635e-4447-8f40-571537bca3f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:18.685770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927a8fa-635e-4447-8f40-571537bca3f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:18.686075 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927a8fa-635e-4447-8f40-571537bca3f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:18.687817 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927a8fa-635e-4447-8f40-571537bca3f6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:18.715372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 833/1666] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:55:18.719632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf94ef-9247-41be-ab6d-068270d662a2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:18.736502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf94ef-9247-41be-ab6d-068270d662a2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:18.745453 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bbf85-abba-4233-b858-266c8ca815cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:18.746509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bbf85-abba-4233-b858-266c8ca815cd admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:18.746916 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bbf85-abba-4233-b858-266c8ca815cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:18.747473 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bbf85-abba-4233-b858-266c8ca815cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:18.748553 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516bbf85-abba-4233-b858-266c8ca815cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:18.750807 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516bbf85-abba-4233-b858-266c8ca815cd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:18.823902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 834/1667] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:18 2022] POST /identity/v3/projects => generated 405 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:18.828001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927a8fa-635e-4447-8f40-571537bca3f6 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:18.846986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927a8fa-635e-4447-8f40-571537bca3f6 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:18.970250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:19.069071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43544fcc-f213-4c96-aaeb-163345686f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:19.069715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43544fcc-f213-4c96-aaeb-163345686f6d admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:19.069997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43544fcc-f213-4c96-aaeb-163345686f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:19.070271 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43544fcc-f213-4c96-aaeb-163345686f6d admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:19.071135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43544fcc-f213-4c96-aaeb-163345686f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:19.073614 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43544fcc-f213-4c96-aaeb-163345686f6d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:19.156856 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bbf85-abba-4233-b858-266c8ca815cd admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:19.163876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 834/1668] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:18 2022] POST /identity/v3/users => generated 303 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:19.171834 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bbf85-abba-4233-b858-266c8ca815cd admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:19.215223 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41aa8bc-f03a-4ea7-b50e-4332cabb7f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:19.215950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41aa8bc-f03a-4ea7-b50e-4332cabb7f29 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:19.216152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41aa8bc-f03a-4ea7-b50e-4332cabb7f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:19.216429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41aa8bc-f03a-4ea7-b50e-4332cabb7f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:19.217083 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41aa8bc-f03a-4ea7-b50e-4332cabb7f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:19.218699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41aa8bc-f03a-4ea7-b50e-4332cabb7f29 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:19.253835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 835/1669] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:55:19.258118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43544fcc-f213-4c96-aaeb-163345686f6d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:19.273704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43544fcc-f213-4c96-aaeb-163345686f6d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:19.291911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380118d-13dc-43fd-aa39-bf8898eab1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:19.292587 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380118d-13dc-43fd-aa39-bf8898eab1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:19.292870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380118d-13dc-43fd-aa39-bf8898eab1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:19.293188 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380118d-13dc-43fd-aa39-bf8898eab1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:19.293862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380118d-13dc-43fd-aa39-bf8898eab1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:19.295875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380118d-13dc-43fd-aa39-bf8898eab1c1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:19.351983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 835/1670] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:19.357015 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41aa8bc-f03a-4ea7-b50e-4332cabb7f29 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:19.376574 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41aa8bc-f03a-4ea7-b50e-4332cabb7f29 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:19.382519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38f128-7b70-451e-8790-61b405ae9768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:19.383169 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38f128-7b70-451e-8790-61b405ae9768 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:19.383447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38f128-7b70-451e-8790-61b405ae9768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:19.383721 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38f128-7b70-451e-8790-61b405ae9768 admin admin] PATH_INFO: `/v3/projects/8a5b8f3bfd45460993e41576399a033e/users/fd5d3de8622e4295b18e790d09217a48/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:19.424366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38f128-7b70-451e-8790-61b405ae9768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5b8f3bfd45460993e41576399a033e, user_id=fd5d3de8622e4295b18e790d09217a48, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:19.428498 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38f128-7b70-451e-8790-61b405ae9768 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:19.455281 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 836/1671] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:19 2022] PUT /identity/v3/projects/8a5b8f3bfd45460993e41576399a033e/users/fd5d3de8622e4295b18e790d09217a48/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:19.460383 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226569d-0f8f-4612-b945-b3d269d07d94 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:19.460672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226569d-0f8f-4612-b945-b3d269d07d94 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:19.460945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226569d-0f8f-4612-b945-b3d269d07d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:19.514930 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-2226569d-0f8f-4612-b945-b3d269d07d94 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:19.521231 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2226569d-0f8f-4612-b945-b3d269d07d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:19.762750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 836/1672] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:19.846196 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38f128-7b70-451e-8790-61b405ae9768 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:19.850535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7d38f128-7b70-451e-8790-61b405ae9768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:20.004098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38f128-7b70-451e-8790-61b405ae9768 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:20.027549 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92136a-39e0-4deb-b21e-2c642d9267bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:20.028221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92136a-39e0-4deb-b21e-2c642d9267bc service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:20.028506 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92136a-39e0-4deb-b21e-2c642d9267bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:20.028781 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92136a-39e0-4deb-b21e-2c642d9267bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:20.051248 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92136a-39e0-4deb-b21e-2c642d9267bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:20.053342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92136a-39e0-4deb-b21e-2c642d9267bc service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:20.080369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 837/1673] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:22.396607 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822acf7-eed5-4834-a46c-00545f8bfd04 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:22.400421 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822acf7-eed5-4834-a46c-00545f8bfd04 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:22.400421 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822acf7-eed5-4834-a46c-00545f8bfd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:22.426030 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92136a-39e0-4deb-b21e-2c642d9267bc service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:22.431829 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4f92136a-39e0-4deb-b21e-2c642d9267bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:22.489201 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92136a-39e0-4deb-b21e-2c642d9267bc service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:22.514090 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007b6ae-ab28-42ac-b11e-4098bea88bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:22.514962 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007b6ae-ab28-42ac-b11e-4098bea88bb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:22.516574 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007b6ae-ab28-42ac-b11e-4098bea88bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:22.517025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007b6ae-ab28-42ac-b11e-4098bea88bb3 admin admin] PATH_INFO: `/v3/users/f2b563a5c1524b8f9cb3ef9d68eca5be` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:22.527923 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007b6ae-ab28-42ac-b11e-4098bea88bb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2b563a5c1524b8f9cb3ef9d68eca5be)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:22.538170 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007b6ae-ab28-42ac-b11e-4098bea88bb3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:22.548631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e822acf7-eed5-4834-a46c-00545f8bfd04 None None] MFA Rules not processed for user `4035909c0f414e5ca7fbab2caee29f4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:22.552595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e822acf7-eed5-4834-a46c-00545f8bfd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:22.687472 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e007b6ae-ab28-42ac-b11e-4098bea88bb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2b563a5c1524b8f9cb3ef9d68eca5be'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:22.726372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:22.740782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e007b6ae-ab28-42ac-b11e-4098bea88bb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2b563a5c1524b8f9cb3ef9d68eca5be'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:22.772493 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e007b6ae-ab28-42ac-b11e-4098bea88bb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2b563a5c1524b8f9cb3ef9d68eca5be'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:22.773464 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e007b6ae-ab28-42ac-b11e-4098bea88bb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2b563a5c1524b8f9cb3ef9d68eca5be'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:22.815843 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e007b6ae-ab28-42ac-b11e-4098bea88bb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2b563a5c1524b8f9cb3ef9d68eca5be'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:22.821208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 837/1674] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:22.824887 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43316936-ed1a-4e9b-8687-ef3649797645 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:22.825374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43316936-ed1a-4e9b-8687-ef3649797645 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:22.825796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43316936-ed1a-4e9b-8687-ef3649797645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:22.852606 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 838/1675] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:22 2022] DELETE /identity/v3/users/f2b563a5c1524b8f9cb3ef9d68eca5be => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:22.859115 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba141bc-cc2a-4e48-a1e1-995e0ebf6b77 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:22.859524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba141bc-cc2a-4e48-a1e1-995e0ebf6b77 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:22.859914 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba141bc-cc2a-4e48-a1e1-995e0ebf6b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:22.860514 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-43316936-ed1a-4e9b-8687-ef3649797645 None None] MFA Rules not processed for user `4035909c0f414e5ca7fbab2caee29f4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:22.862505 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-43316936-ed1a-4e9b-8687-ef3649797645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:22.902998 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7ba141bc-cc2a-4e48-a1e1-995e0ebf6b77 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:22.904909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7ba141bc-cc2a-4e48-a1e1-995e0ebf6b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:22.950367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:23.039651 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 838/1676] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:23.041959 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43316936-ed1a-4e9b-8687-ef3649797645 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:23.045185 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-43316936-ed1a-4e9b-8687-ef3649797645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:23.083904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 839/1677] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:23.126094 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba141bc-cc2a-4e48-a1e1-995e0ebf6b77 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:23.130394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7ba141bc-cc2a-4e48-a1e1-995e0ebf6b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:23.239482 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43316936-ed1a-4e9b-8687-ef3649797645 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:23.252299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607a390-64e1-4e78-bcfb-fe6f8fcf2f49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:23.252923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607a390-64e1-4e78-bcfb-fe6f8fcf2f49 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:23.253208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607a390-64e1-4e78-bcfb-fe6f8fcf2f49 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:23.253484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607a390-64e1-4e78-bcfb-fe6f8fcf2f49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:23.260548 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c607a390-64e1-4e78-bcfb-fe6f8fcf2f49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:23.265191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba141bc-cc2a-4e48-a1e1-995e0ebf6b77 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:23.280502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f648cef-9f5f-45bf-8b91-297d8e7e53fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:23.281290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f648cef-9f5f-45bf-8b91-297d8e7e53fe service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:23.281698 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f648cef-9f5f-45bf-8b91-297d8e7e53fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:23.282092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f648cef-9f5f-45bf-8b91-297d8e7e53fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:23.286389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607a390-64e1-4e78-bcfb-fe6f8fcf2f49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:23.293266 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607a390-64e1-4e78-bcfb-fe6f8fcf2f49 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:23.303067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f648cef-9f5f-45bf-8b91-297d8e7e53fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:23.304853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f648cef-9f5f-45bf-8b91-297d8e7e53fe service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:23.316256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 839/1678] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:23.345364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 840/1679] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:24.075265 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607a390-64e1-4e78-bcfb-fe6f8fcf2f49 service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:24.076972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c607a390-64e1-4e78-bcfb-fe6f8fcf2f49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:24.099071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607a390-64e1-4e78-bcfb-fe6f8fcf2f49 service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:24.105499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050581a4-faf4-49e5-a4f9-8bce15cf90e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:24.107139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050581a4-faf4-49e5-a4f9-8bce15cf90e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:24.107844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050581a4-faf4-49e5-a4f9-8bce15cf90e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:24.108412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050581a4-faf4-49e5-a4f9-8bce15cf90e4 admin admin] PATH_INFO: `/v3/projects/7a829819301a4a5caa00a85ae0e4fe10` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:24.110615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050581a4-faf4-49e5-a4f9-8bce15cf90e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a829819301a4a5caa00a85ae0e4fe10)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:24.125584 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050581a4-faf4-49e5-a4f9-8bce15cf90e4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:24.230546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-050581a4-faf4-49e5-a4f9-8bce15cf90e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a829819301a4a5caa00a85ae0e4fe10', 'request_id': 'req-050581a4-faf4-49e5-a4f9-8bce15cf90e4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:24.240577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-050581a4-faf4-49e5-a4f9-8bce15cf90e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a829819301a4a5caa00a85ae0e4fe10', 'request_id': 'req-050581a4-faf4-49e5-a4f9-8bce15cf90e4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:24.251161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-050581a4-faf4-49e5-a4f9-8bce15cf90e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a829819301a4a5caa00a85ae0e4fe10', 'request_id': 'req-050581a4-faf4-49e5-a4f9-8bce15cf90e4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:24.280188 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-050581a4-faf4-49e5-a4f9-8bce15cf90e4 admin admin] The token cache is being invalidate because project 7a829819301a4a5caa00a85ae0e4fe10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:24.281267 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-050581a4-faf4-49e5-a4f9-8bce15cf90e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:24.282131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-050581a4-faf4-49e5-a4f9-8bce15cf90e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:24.284658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 840/1680] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:24 2022] DELETE /identity/v3/projects/7a829819301a4a5caa00a85ae0e4fe10 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:24.289896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d059120-5276-4ffc-a79e-16ed9fbc23bc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:24.290525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d059120-5276-4ffc-a79e-16ed9fbc23bc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:24.291582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d059120-5276-4ffc-a79e-16ed9fbc23bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:24.324934 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7d059120-5276-4ffc-a79e-16ed9fbc23bc None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:24.326928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7d059120-5276-4ffc-a79e-16ed9fbc23bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:24.452709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 841/1681] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:55:24.460455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4348b-e782-49a3-93cd-03549e7b50e8 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:24.460897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4348b-e782-49a3-93cd-03549e7b50e8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:24.461328 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4348b-e782-49a3-93cd-03549e7b50e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:24.492889 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9cc4348b-e782-49a3-93cd-03549e7b50e8 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:24.496589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9cc4348b-e782-49a3-93cd-03549e7b50e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:24.517023 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 841/1682] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:55:24.519284 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d059120-5276-4ffc-a79e-16ed9fbc23bc None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:24.547049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d059120-5276-4ffc-a79e-16ed9fbc23bc None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:24.556597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549a701-2072-43d6-ba47-caf0c5f201b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:24.557177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549a701-2072-43d6-ba47-caf0c5f201b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:24.557456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549a701-2072-43d6-ba47-caf0c5f201b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:24.557726 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549a701-2072-43d6-ba47-caf0c5f201b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:24.558669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549a701-2072-43d6-ba47-caf0c5f201b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:24.558669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549a701-2072-43d6-ba47-caf0c5f201b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:24.560343 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549a701-2072-43d6-ba47-caf0c5f201b4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:24.594988 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 842/1683] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:24.598188 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4348b-e782-49a3-93cd-03549e7b50e8 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:24.632777 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4348b-e782-49a3-93cd-03549e7b50e8 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:24.657309 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709bdac-88c3-4bc6-90fb-6eb4350531a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:24.661825 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709bdac-88c3-4bc6-90fb-6eb4350531a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:24.662208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709bdac-88c3-4bc6-90fb-6eb4350531a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:24.662546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6be14c-52ef-4325-b4e1-978f714a5bc2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:24.662657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709bdac-88c3-4bc6-90fb-6eb4350531a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:24.662838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6be14c-52ef-4325-b4e1-978f714a5bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:24.663113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6be14c-52ef-4325-b4e1-978f714a5bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:24.663409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709bdac-88c3-4bc6-90fb-6eb4350531a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:24.668325 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709bdac-88c3-4bc6-90fb-6eb4350531a0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:24.698263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:24.737290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5e6be14c-52ef-4325-b4e1-978f714a5bc2 None None] MFA Rules not processed for user `fd5d3de8622e4295b18e790d09217a48`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:24.738550 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5e6be14c-52ef-4325-b4e1-978f714a5bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:24.753602 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 842/1684] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:24 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:24.757799 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709bdac-88c3-4bc6-90fb-6eb4350531a0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:24.797434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:24.848565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709bdac-88c3-4bc6-90fb-6eb4350531a0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:24.856834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11ad2a-c453-43bd-9841-cb55560b9e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:24.857464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11ad2a-c453-43bd-9841-cb55560b9e57 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:24.857763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11ad2a-c453-43bd-9841-cb55560b9e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:24.858082 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11ad2a-c453-43bd-9841-cb55560b9e57 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:24.863804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c11ad2a-c453-43bd-9841-cb55560b9e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:24.865339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c11ad2a-c453-43bd-9841-cb55560b9e57 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:24.898066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 843/1685] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:24 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:24.900067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12c478-c7e3-4ae3-a8f4-11c091e56101 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:24.900250 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12c478-c7e3-4ae3-a8f4-11c091e56101 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:24.900662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12c478-c7e3-4ae3-a8f4-11c091e56101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:24.921247 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 843/1686] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:24 2022] POST /identity/v3/users => generated 290 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:24.922294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11ad2a-c453-43bd-9841-cb55560b9e57 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:24.924508 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6c11ad2a-c453-43bd-9841-cb55560b9e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:24.956631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11ad2a-c453-43bd-9841-cb55560b9e57 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:24.961670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0f12c478-c7e3-4ae3-a8f4-11c091e56101 None None] MFA Rules not processed for user `b52ececfc19b41b5b8387dc6908d4a4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:24.963545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0f12c478-c7e3-4ae3-a8f4-11c091e56101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:24.966085 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89382753-8441-4928-bbbe-1c02a4a4ec85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:24.967315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89382753-8441-4928-bbbe-1c02a4a4ec85 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:24.967630 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89382753-8441-4928-bbbe-1c02a4a4ec85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:24.967905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89382753-8441-4928-bbbe-1c02a4a4ec85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:24.972003 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89382753-8441-4928-bbbe-1c02a4a4ec85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:24.976880 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89382753-8441-4928-bbbe-1c02a4a4ec85 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:25.051677 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 844/1687] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:24 2022] POST /identity/v3/projects => generated 403 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:25.053438 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89382753-8441-4928-bbbe-1c02a4a4ec85 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:25.068016 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 844/1688] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:25.069287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12c478-c7e3-4ae3-a8f4-11c091e56101 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:25.087623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12c478-c7e3-4ae3-a8f4-11c091e56101 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:25.235443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ba787-641b-41b6-87bf-e9fa52f56b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:25.237363 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ba787-641b-41b6-87bf-e9fa52f56b51 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:25.238356 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ba787-641b-41b6-87bf-e9fa52f56b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:25.239057 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ba787-641b-41b6-87bf-e9fa52f56b51 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:25.240735 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89382753-8441-4928-bbbe-1c02a4a4ec85 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:25.241609 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ba787-641b-41b6-87bf-e9fa52f56b51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:25.243892 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ba787-641b-41b6-87bf-e9fa52f56b51 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:25.252840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae0daf-a7e7-4348-a066-3f409bf469c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:25.255243 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae0daf-a7e7-4348-a066-3f409bf469c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:25.255635 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae0daf-a7e7-4348-a066-3f409bf469c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:25.255956 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae0daf-a7e7-4348-a066-3f409bf469c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:25.257851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae0daf-a7e7-4348-a066-3f409bf469c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:25.260843 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae0daf-a7e7-4348-a066-3f409bf469c7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:25.298049 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 845/1689] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:25.299675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370324be-7605-4ae3-acab-2160cb8dfae8 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:25.299969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370324be-7605-4ae3-acab-2160cb8dfae8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:25.300274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370324be-7605-4ae3-acab-2160cb8dfae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:25.306084 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 845/1690] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:25 2022] POST /identity/v3/users => generated 308 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:25.306827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ba787-641b-41b6-87bf-e9fa52f56b51 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:25.350341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ba787-641b-41b6-87bf-e9fa52f56b51 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:25.353041 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-370324be-7605-4ae3-acab-2160cb8dfae8 None None] MFA Rules not processed for user `b52ececfc19b41b5b8387dc6908d4a4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:25.354300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-370324be-7605-4ae3-acab-2160cb8dfae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:25.371332 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b5743-d710-4f34-8e6a-c831977d6a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:25.373948 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b5743-d710-4f34-8e6a-c831977d6a1f admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:25.374225 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b5743-d710-4f34-8e6a-c831977d6a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:25.374512 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b5743-d710-4f34-8e6a-c831977d6a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:25.388328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b5743-d710-4f34-8e6a-c831977d6a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:25.390166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b5743-d710-4f34-8e6a-c831977d6a1f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:25.419604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 846/1691] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:25.419604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b5743-d710-4f34-8e6a-c831977d6a1f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:25.432721 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:25.450732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b5743-d710-4f34-8e6a-c831977d6a1f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:25.461743 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a49fc-9d52-497f-9c9f-6623c69ead88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:25.465645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a49fc-9d52-497f-9c9f-6623c69ead88 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:25.466012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a49fc-9d52-497f-9c9f-6623c69ead88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:25.468371 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a49fc-9d52-497f-9c9f-6623c69ead88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:25.469155 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a49fc-9d52-497f-9c9f-6623c69ead88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:25.472287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a49fc-9d52-497f-9c9f-6623c69ead88 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:25.508485 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 847/1692] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:25.515190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a49fc-9d52-497f-9c9f-6623c69ead88 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:25.553276 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 846/1693] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:25.557253 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370324be-7605-4ae3-acab-2160cb8dfae8 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:25.601192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a49fc-9d52-497f-9c9f-6623c69ead88 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:25.610210 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521459d0-1337-4ee3-9bbc-9739e481834b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:25.611214 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521459d0-1337-4ee3-9bbc-9739e481834b admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:25.611675 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521459d0-1337-4ee3-9bbc-9739e481834b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:25.612197 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521459d0-1337-4ee3-9bbc-9739e481834b admin admin] PATH_INFO: `/v3/projects/c61fd96b94b8426a803325ed9257fc4d/users/3841bac8b11e4f70ade14c7e9132793c/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:25.613840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370324be-7605-4ae3-acab-2160cb8dfae8 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:25.623017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bec85-befd-4879-bc8f-f6096f6b615a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:25.624008 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bec85-befd-4879-bc8f-f6096f6b615a admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:25.624316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bec85-befd-4879-bc8f-f6096f6b615a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:25.624924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bec85-befd-4879-bc8f-f6096f6b615a admin admin] PATH_INFO: `/v3/projects/644c9956bae842c7a6f0ce40d2a94591/users/299b501213f5480992bc8f57e049573b/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:25.649183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521459d0-1337-4ee3-9bbc-9739e481834b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61fd96b94b8426a803325ed9257fc4d, user_id=3841bac8b11e4f70ade14c7e9132793c, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:25.655561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521459d0-1337-4ee3-9bbc-9739e481834b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:25.672562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352bec85-befd-4879-bc8f-f6096f6b615a admin admin] RBAC: Authorizing `identity:create_grant(project_id=644c9956bae842c7a6f0ce40d2a94591, user_id=299b501213f5480992bc8f57e049573b, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:25.678058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352bec85-befd-4879-bc8f-f6096f6b615a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:25.678240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 848/1694] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:25 2022] PUT /identity/v3/projects/c61fd96b94b8426a803325ed9257fc4d/users/3841bac8b11e4f70ade14c7e9132793c/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:25.679967 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63a61f-4330-4a49-a888-5e984240ab61 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:25.680559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63a61f-4330-4a49-a888-5e984240ab61 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:25.681557 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63a61f-4330-4a49-a888-5e984240ab61 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:25.683247 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 849/1695] 10.222.0.26 () {54 vars in 912 bytes} [Wed Sep 7 07:55:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:55:25.684185 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63a61f-4330-4a49-a888-5e984240ab61 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:25.686331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4d63a61f-4330-4a49-a888-5e984240ab61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:25.716163 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 847/1696] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:25 2022] PUT /identity/v3/projects/644c9956bae842c7a6f0ce40d2a94591/users/299b501213f5480992bc8f57e049573b/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:25.717654 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c40b-2f51-4ea4-993e-d4e295a6245d None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:25.717943 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c40b-2f51-4ea4-993e-d4e295a6245d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:25.720438 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c40b-2f51-4ea4-993e-d4e295a6245d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:25.766085 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:25.793170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e682c40b-2f51-4ea4-993e-d4e295a6245d None None] MFA Rules not processed for user `3841bac8b11e4f70ade14c7e9132793c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:25.796290 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e682c40b-2f51-4ea4-993e-d4e295a6245d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:25.877234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63a61f-4330-4a49-a888-5e984240ab61 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:26.051273 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 848/1697] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:26.053963 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783540e7-e000-4b90-9894-a9655839a1e5 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:26.057622 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783540e7-e000-4b90-9894-a9655839a1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:26.058459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783540e7-e000-4b90-9894-a9655839a1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:26.076485 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594625c-10fd-4fa7-ae17-ecbdeec6c959 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:26.083248 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594625c-10fd-4fa7-ae17-ecbdeec6c959 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:26.084200 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594625c-10fd-4fa7-ae17-ecbdeec6c959 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:26.086295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594625c-10fd-4fa7-ae17-ecbdeec6c959 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:26.132469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-783540e7-e000-4b90-9894-a9655839a1e5 None None] MFA Rules not processed for user `330a38822d33439191773e9a8b4cbe04`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:26.148953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-783540e7-e000-4b90-9894-a9655839a1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:26.247548 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594625c-10fd-4fa7-ae17-ecbdeec6c959 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:26.249763 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594625c-10fd-4fa7-ae17-ecbdeec6c959 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:26.292729 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 850/1698] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:26.293742 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594625c-10fd-4fa7-ae17-ecbdeec6c959 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:26.307705 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:26.360128 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 849/1699] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Sep 7 07:55:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:26.365277 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783540e7-e000-4b90-9894-a9655839a1e5 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:26.494123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594625c-10fd-4fa7-ae17-ecbdeec6c959 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:26.507832 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc84f5-699b-4aa0-9d10-e54a07750d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:26.511653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc84f5-699b-4aa0-9d10-e54a07750d0d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:26.512234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc84f5-699b-4aa0-9d10-e54a07750d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:26.515258 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc84f5-699b-4aa0-9d10-e54a07750d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:26.516043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc84f5-699b-4aa0-9d10-e54a07750d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:26.518838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc84f5-699b-4aa0-9d10-e54a07750d0d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:26.555615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 851/1700] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:26.560192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc84f5-699b-4aa0-9d10-e54a07750d0d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:26.568890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1acc84f5-699b-4aa0-9d10-e54a07750d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:26.608891 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc84f5-699b-4aa0-9d10-e54a07750d0d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:26.619758 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426addbf-6ab6-4bad-8fcc-7a8ffaec6bf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:26.621074 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426addbf-6ab6-4bad-8fcc-7a8ffaec6bf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:26.621074 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426addbf-6ab6-4bad-8fcc-7a8ffaec6bf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:26.621074 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426addbf-6ab6-4bad-8fcc-7a8ffaec6bf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:26.636478 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783540e7-e000-4b90-9894-a9655839a1e5 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:26.651441 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426addbf-6ab6-4bad-8fcc-7a8ffaec6bf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:26.651658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c1c4b-6949-49b5-b587-23e92adf138b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:26.653124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c1c4b-6949-49b5-b587-23e92adf138b admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:26.659221 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c1c4b-6949-49b5-b587-23e92adf138b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:26.659887 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c1c4b-6949-49b5-b587-23e92adf138b admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:26.666366 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c1c4b-6949-49b5-b587-23e92adf138b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:26.671416 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426addbf-6ab6-4bad-8fcc-7a8ffaec6bf0 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:26.675876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c1c4b-6949-49b5-b587-23e92adf138b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:26.704212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 852/1701] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:26.705190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426addbf-6ab6-4bad-8fcc-7a8ffaec6bf0 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:26.734430 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426addbf-6ab6-4bad-8fcc-7a8ffaec6bf0 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:26.748766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacbee1-3698-484d-8b02-21f7186fe349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:26.749743 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacbee1-3698-484d-8b02-21f7186fe349 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:26.750488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacbee1-3698-484d-8b02-21f7186fe349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:26.750763 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacbee1-3698-484d-8b02-21f7186fe349 admin admin] PATH_INFO: `/v3/projects/644c9956bae842c7a6f0ce40d2a94591/users/299b501213f5480992bc8f57e049573b/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:26.757489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacbee1-3698-484d-8b02-21f7186fe349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=644c9956bae842c7a6f0ce40d2a94591, user_id=299b501213f5480992bc8f57e049573b, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:26.762503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacbee1-3698-484d-8b02-21f7186fe349 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:26.781832 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 850/1702] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:26 2022] POST /identity/v3/projects => generated 379 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:26.781832 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c1c4b-6949-49b5-b587-23e92adf138b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:26.781832 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d72c1c4b-6949-49b5-b587-23e92adf138b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:26.800919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 853/1703] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:26 2022] PUT /identity/v3/projects/644c9956bae842c7a6f0ce40d2a94591/users/299b501213f5480992bc8f57e049573b/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:26.802001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacbee1-3698-484d-8b02-21f7186fe349 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:26.843347 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:26.963313 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacbee1-3698-484d-8b02-21f7186fe349 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:26.982241 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c1c4b-6949-49b5-b587-23e92adf138b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:26.985310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae0215-edd6-43de-8d89-0156bb7babf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:26.985927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae0215-edd6-43de-8d89-0156bb7babf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:26.986208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae0215-edd6-43de-8d89-0156bb7babf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:26.986479 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae0215-edd6-43de-8d89-0156bb7babf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:26.987250 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae0215-edd6-43de-8d89-0156bb7babf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:26.989267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae0215-edd6-43de-8d89-0156bb7babf4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:27.001322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a52712-1435-4df8-a039-fb533b100f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:27.002086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a52712-1435-4df8-a039-fb533b100f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:27.002466 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a52712-1435-4df8-a039-fb533b100f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:27.002862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a52712-1435-4df8-a039-fb533b100f60 admin admin] PATH_INFO: `/v3/users/0778bbd644bf4b2ea70a8031ebb12783` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:27.004142 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a52712-1435-4df8-a039-fb533b100f60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0778bbd644bf4b2ea70a8031ebb12783)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:27.006189 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a52712-1435-4df8-a039-fb533b100f60 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:27.042074 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 854/1704] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:26 2022] POST /identity/v3/users => generated 296 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:27.046782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae0215-edd6-43de-8d89-0156bb7babf4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:27.085219 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-29a52712-1435-4df8-a039-fb533b100f60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0778bbd644bf4b2ea70a8031ebb12783'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:27.085470 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae0215-edd6-43de-8d89-0156bb7babf4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:27.086419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-29a52712-1435-4df8-a039-fb533b100f60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0778bbd644bf4b2ea70a8031ebb12783'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:27.105923 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac220cc-c77f-4d4f-829c-043705536073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:27.106545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac220cc-c77f-4d4f-829c-043705536073 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:27.106825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac220cc-c77f-4d4f-829c-043705536073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:27.107252 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac220cc-c77f-4d4f-829c-043705536073 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:27.107868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-29a52712-1435-4df8-a039-fb533b100f60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0778bbd644bf4b2ea70a8031ebb12783'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:27.107987 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac220cc-c77f-4d4f-829c-043705536073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:27.109275 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-29a52712-1435-4df8-a039-fb533b100f60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0778bbd644bf4b2ea70a8031ebb12783'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:27.109514 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac220cc-c77f-4d4f-829c-043705536073 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:27.119417 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-29a52712-1435-4df8-a039-fb533b100f60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0778bbd644bf4b2ea70a8031ebb12783'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:27.136563 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 855/1705] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:55:27.139745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 851/1706] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:26 2022] DELETE /identity/v3/users/0778bbd644bf4b2ea70a8031ebb12783 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:27.145094 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac220cc-c77f-4d4f-829c-043705536073 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:27.145929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935fe82-4f85-4835-97cf-58d9003e61b2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:27.146232 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935fe82-4f85-4835-97cf-58d9003e61b2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:27.147646 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935fe82-4f85-4835-97cf-58d9003e61b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:27.152273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-aac220cc-c77f-4d4f-829c-043705536073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:27.176240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac220cc-c77f-4d4f-829c-043705536073 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:27.180140 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-0935fe82-4f85-4835-97cf-58d9003e61b2 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:27.181926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0935fe82-4f85-4835-97cf-58d9003e61b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:27.311328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:27.342936 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509df80-331d-44b1-b9dd-26b327a17aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:27.342936 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509df80-331d-44b1-b9dd-26b327a17aef admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:27.342936 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509df80-331d-44b1-b9dd-26b327a17aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:27.342936 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509df80-331d-44b1-b9dd-26b327a17aef admin admin] PATH_INFO: `/v3/projects/3e471b8be3f94808851cb4c13608dfc0/users/3fa4c6e147f24fa298c09626ecd100ae/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:27.346151 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 852/1707] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:27.393423 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509df80-331d-44b1-b9dd-26b327a17aef admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e471b8be3f94808851cb4c13608dfc0, user_id=3fa4c6e147f24fa298c09626ecd100ae, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:27.403497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509df80-331d-44b1-b9dd-26b327a17aef admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:27.469741 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 856/1708] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:27 2022] PUT /identity/v3/projects/3e471b8be3f94808851cb4c13608dfc0/users/3fa4c6e147f24fa298c09626ecd100ae/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:27.472630 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935fe82-4f85-4835-97cf-58d9003e61b2 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:27.496699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b53802-e940-41bd-8ec9-d080210cff0e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:27.497277 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b53802-e940-41bd-8ec9-d080210cff0e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:27.497806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b53802-e940-41bd-8ec9-d080210cff0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:27.524868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935fe82-4f85-4835-97cf-58d9003e61b2 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:27.664376 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-46b53802-e940-41bd-8ec9-d080210cff0e None None] MFA Rules not processed for user `e07bc40275734f628fcc2f62dd4dd0e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:27.668056 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-46b53802-e940-41bd-8ec9-d080210cff0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:27.715121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:27.773950 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763d841-15d2-4ca6-ba89-4658bfb9a6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:27.775318 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763d841-15d2-4ca6-ba89-4658bfb9a6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:27.775696 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763d841-15d2-4ca6-ba89-4658bfb9a6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:27.776329 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763d841-15d2-4ca6-ba89-4658bfb9a6c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:27.778157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763d841-15d2-4ca6-ba89-4658bfb9a6c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:27.781132 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763d841-15d2-4ca6-ba89-4658bfb9a6c2 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:27.822885 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 853/1709] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:27.822885 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763d841-15d2-4ca6-ba89-4658bfb9a6c2 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:27.824153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f763d841-15d2-4ca6-ba89-4658bfb9a6c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:27.897331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 857/1710] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:27.903674 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac155c-c46b-4622-a6ea-ef93ef7725a8 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:27.905376 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac155c-c46b-4622-a6ea-ef93ef7725a8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:27.906700 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac155c-c46b-4622-a6ea-ef93ef7725a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:27.954474 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2bac155c-c46b-4622-a6ea-ef93ef7725a8 None None] MFA Rules not processed for user `457d69a7306d46cd8445f887eb95c517`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:27.956771 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2bac155c-c46b-4622-a6ea-ef93ef7725a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:27.999654 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763d841-15d2-4ca6-ba89-4658bfb9a6c2 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:28.004253 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b8b65-e269-404c-96af-e66d036db8f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:28.005395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b8b65-e269-404c-96af-e66d036db8f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:28.005395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b8b65-e269-404c-96af-e66d036db8f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:28.005395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b8b65-e269-404c-96af-e66d036db8f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:28.022737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b8b65-e269-404c-96af-e66d036db8f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:28.024008 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b8b65-e269-404c-96af-e66d036db8f9 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:28.062193 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 854/1711] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:28.065459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dfdc0-3b81-4236-b2d0-7e1c6157d798 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:28.067016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dfdc0-3b81-4236-b2d0-7e1c6157d798 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:28.067695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dfdc0-3b81-4236-b2d0-7e1c6157d798 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:28.074774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 858/1712] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:28.075884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac155c-c46b-4622-a6ea-ef93ef7725a8 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:28.075988 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 855/1713] 10.222.0.26 () {54 vars in 912 bytes} [Wed Sep 7 07:55:28 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:55:28.077213 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dfdc0-3b81-4236-b2d0-7e1c6157d798 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:28.079955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3c0dfdc0-3b81-4236-b2d0-7e1c6157d798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:28.105086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac155c-c46b-4622-a6ea-ef93ef7725a8 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:28.116817 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f6179-beb5-4610-931a-63f2312af5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:28.117695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f6179-beb5-4610-931a-63f2312af5fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:28.118750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f6179-beb5-4610-931a-63f2312af5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:28.119302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f6179-beb5-4610-931a-63f2312af5fe admin admin] PATH_INFO: `/v3/projects/3e471b8be3f94808851cb4c13608dfc0/users/3fa4c6e147f24fa298c09626ecd100ae/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:28.121773 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f6179-beb5-4610-931a-63f2312af5fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e471b8be3f94808851cb4c13608dfc0, user_id=3fa4c6e147f24fa298c09626ecd100ae, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:28.125384 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f6179-beb5-4610-931a-63f2312af5fe admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:28.145142 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 859/1714] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:28 2022] PUT /identity/v3/projects/3e471b8be3f94808851cb4c13608dfc0/users/3fa4c6e147f24fa298c09626ecd100ae/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:28.146826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104d1b0-02dc-4c6b-9724-5f53c24be8b5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:28.149987 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104d1b0-02dc-4c6b-9724-5f53c24be8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:28.150453 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104d1b0-02dc-4c6b-9724-5f53c24be8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:28.198467 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8104d1b0-02dc-4c6b-9724-5f53c24be8b5 None None] MFA Rules not processed for user `330a38822d33439191773e9a8b4cbe04`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:28.199811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8104d1b0-02dc-4c6b-9724-5f53c24be8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:28.256474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dfdc0-3b81-4236-b2d0-7e1c6157d798 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:28.262997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:28.286778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 860/1715] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Sep 7 07:55:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:28.288235 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104d1b0-02dc-4c6b-9724-5f53c24be8b5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:28.290593 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8104d1b0-02dc-4c6b-9724-5f53c24be8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:28.369636 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94e40e-af69-4b6e-98bd-728f5629de32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:28.370261 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94e40e-af69-4b6e-98bd-728f5629de32 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:28.370541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94e40e-af69-4b6e-98bd-728f5629de32 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:28.370848 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94e40e-af69-4b6e-98bd-728f5629de32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:28.384752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104d1b0-02dc-4c6b-9724-5f53c24be8b5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:28.393061 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc3e97-93c0-401b-8883-2b0890ef6fee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:28.394154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc3e97-93c0-401b-8883-2b0890ef6fee service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:28.394716 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc3e97-93c0-401b-8883-2b0890ef6fee service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:28.395233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc3e97-93c0-401b-8883-2b0890ef6fee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:28.397636 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:28.397918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-31dc3e97-93c0-401b-8883-2b0890ef6fee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:28.424169 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc3e97-93c0-401b-8883-2b0890ef6fee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:28.429592 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc3e97-93c0-401b-8883-2b0890ef6fee service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:28.449642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 861/1716] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:28.451956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669c5a1-c77f-4050-a6d9-df4bb3159667 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:28.452821 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669c5a1-c77f-4050-a6d9-df4bb3159667 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:28.453227 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669c5a1-c77f-4050-a6d9-df4bb3159667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:28.518406 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94e40e-af69-4b6e-98bd-728f5629de32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:28.520227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94e40e-af69-4b6e-98bd-728f5629de32 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:28.549304 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 856/1717] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:28.553057 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7669c5a1-c77f-4050-a6d9-df4bb3159667 None None] MFA Rules not processed for user `3fa4c6e147f24fa298c09626ecd100ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:28.554315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94e40e-af69-4b6e-98bd-728f5629de32 service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:28.557105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7669c5a1-c77f-4050-a6d9-df4bb3159667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:28.581761 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94e40e-af69-4b6e-98bd-728f5629de32 service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:28.594675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330b10b-2281-4312-b05e-fc29a8f601a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:28.595512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330b10b-2281-4312-b05e-fc29a8f601a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:28.596147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330b10b-2281-4312-b05e-fc29a8f601a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:28.596625 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330b10b-2281-4312-b05e-fc29a8f601a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:28.618086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330b10b-2281-4312-b05e-fc29a8f601a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:28.627901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330b10b-2281-4312-b05e-fc29a8f601a9 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:28.674335 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 857/1718] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:28.832507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 862/1719] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:28.848685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330b10b-2281-4312-b05e-fc29a8f601a9 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:28.850715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b330b10b-2281-4312-b05e-fc29a8f601a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:29.067618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330b10b-2281-4312-b05e-fc29a8f601a9 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:29.091899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159dff5-8d19-45f2-a243-250cc82893f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:29.095966 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159dff5-8d19-45f2-a243-250cc82893f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:29.096283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159dff5-8d19-45f2-a243-250cc82893f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:29.096564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159dff5-8d19-45f2-a243-250cc82893f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:29.159026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159dff5-8d19-45f2-a243-250cc82893f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:29.161030 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159dff5-8d19-45f2-a243-250cc82893f7 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:29.164542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669c5a1-c77f-4050-a6d9-df4bb3159667 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:29.190098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669c5a1-c77f-4050-a6d9-df4bb3159667 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:29.198816 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 858/1720] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:29.204432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742865e-06f0-4cf3-a29f-d2fba6f18b7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:29.206140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742865e-06f0-4cf3-a29f-d2fba6f18b7b service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:29.207117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742865e-06f0-4cf3-a29f-d2fba6f18b7b service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:29.207422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742865e-06f0-4cf3-a29f-d2fba6f18b7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:29.350262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159dff5-8d19-45f2-a243-250cc82893f7 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:29.362326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159dff5-8d19-45f2-a243-250cc82893f7 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:29.386574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea87da-ca48-413a-a729-69d37afefa1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:29.386574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea87da-ca48-413a-a729-69d37afefa1a service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:29.386780 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea87da-ca48-413a-a729-69d37afefa1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:29.387038 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea87da-ca48-413a-a729-69d37afefa1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:29.392626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-17ea87da-ca48-413a-a729-69d37afefa1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:29.445062 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742865e-06f0-4cf3-a29f-d2fba6f18b7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:29.450799 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742865e-06f0-4cf3-a29f-d2fba6f18b7b service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:29.492171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 863/1721] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:29 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:29.625704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea87da-ca48-413a-a729-69d37afefa1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:29.627198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea87da-ca48-413a-a729-69d37afefa1a service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:29.667437 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 859/1722] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:30.090256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742865e-06f0-4cf3-a29f-d2fba6f18b7b service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:30.092184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8742865e-06f0-4cf3-a29f-d2fba6f18b7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:30.102941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:30.216224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742865e-06f0-4cf3-a29f-d2fba6f18b7b service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:30.231666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b5ded-fd70-4614-af15-689e431bc72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:30.233125 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b5ded-fd70-4614-af15-689e431bc72d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:30.234066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b5ded-fd70-4614-af15-689e431bc72d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:30.234898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b5ded-fd70-4614-af15-689e431bc72d admin admin] PATH_INFO: `/v3/projects/1ded590044ea49dfba66ab456d831fe1` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:30.248400 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b5ded-fd70-4614-af15-689e431bc72d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ded590044ea49dfba66ab456d831fe1)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:30.261124 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b5ded-fd70-4614-af15-689e431bc72d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:30.355177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8c3b5ded-fd70-4614-af15-689e431bc72d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ded590044ea49dfba66ab456d831fe1', 'request_id': 'req-8c3b5ded-fd70-4614-af15-689e431bc72d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:30.375053 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8c3b5ded-fd70-4614-af15-689e431bc72d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ded590044ea49dfba66ab456d831fe1', 'request_id': 'req-8c3b5ded-fd70-4614-af15-689e431bc72d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:30.395850 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8c3b5ded-fd70-4614-af15-689e431bc72d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ded590044ea49dfba66ab456d831fe1', 'request_id': 'req-8c3b5ded-fd70-4614-af15-689e431bc72d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:30.436005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8c3b5ded-fd70-4614-af15-689e431bc72d admin admin] The token cache is being invalidate because project 1ded590044ea49dfba66ab456d831fe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:30.436334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8c3b5ded-fd70-4614-af15-689e431bc72d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:30.442375 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8c3b5ded-fd70-4614-af15-689e431bc72d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:30.448585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 864/1723] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:30 2022] DELETE /identity/v3/projects/1ded590044ea49dfba66ab456d831fe1 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:30.451712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea87da-ca48-413a-a729-69d37afefa1a service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:30.476041 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea87da-ca48-413a-a729-69d37afefa1a service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:30.671040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:30.703882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f22118-c5bf-4a96-98fc-2dd996ae0553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:30.707390 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f22118-c5bf-4a96-98fc-2dd996ae0553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:30.708013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f22118-c5bf-4a96-98fc-2dd996ae0553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:30.708761 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f22118-c5bf-4a96-98fc-2dd996ae0553 admin admin] PATH_INFO: `/v3/users/2a7297b50b834fc29399341518bb22d2` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:30.710268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f22118-c5bf-4a96-98fc-2dd996ae0553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a7297b50b834fc29399341518bb22d2)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:30.712593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f22118-c5bf-4a96-98fc-2dd996ae0553 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:30.806028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-33f22118-c5bf-4a96-98fc-2dd996ae0553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a7297b50b834fc29399341518bb22d2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:30.812922 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-33f22118-c5bf-4a96-98fc-2dd996ae0553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a7297b50b834fc29399341518bb22d2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:30.845946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-33f22118-c5bf-4a96-98fc-2dd996ae0553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a7297b50b834fc29399341518bb22d2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:30.849016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-33f22118-c5bf-4a96-98fc-2dd996ae0553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a7297b50b834fc29399341518bb22d2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:30.887442 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-33f22118-c5bf-4a96-98fc-2dd996ae0553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a7297b50b834fc29399341518bb22d2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:30.900357 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1c052-d2c1-425d-a9f7-d53c440a50df None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:30.901473 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1c052-d2c1-425d-a9f7-d53c440a50df None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:30.901803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1c052-d2c1-425d-a9f7-d53c440a50df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:30.941353 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 860/1724] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:30 2022] DELETE /identity/v3/users/2a7297b50b834fc29399341518bb22d2 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:31.038425 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55fb67-657a-4b12-af3e-b91245012b3e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:31.038711 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55fb67-657a-4b12-af3e-b91245012b3e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:31.038982 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55fb67-657a-4b12-af3e-b91245012b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:31.051256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-34e1c052-d2c1-425d-a9f7-d53c440a50df None None] MFA Rules not processed for user `299b501213f5480992bc8f57e049573b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:31.052946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-34e1c052-d2c1-425d-a9f7-d53c440a50df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:31.165595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-4d55fb67-657a-4b12-af3e-b91245012b3e None None] MFA Rules not processed for user `b52ececfc19b41b5b8387dc6908d4a4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:31.169881 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4d55fb67-657a-4b12-af3e-b91245012b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:31.293872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 865/1725] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:30 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:31.442568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 861/1726] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:55:33.136104 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f73dae-a858-496b-877f-b1cad455db38 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:33.137817 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f73dae-a858-496b-877f-b1cad455db38 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:33.138101 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f73dae-a858-496b-877f-b1cad455db38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:33.152787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55fb67-657a-4b12-af3e-b91245012b3e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:33.155115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4d55fb67-657a-4b12-af3e-b91245012b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:33.200711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:33.278848 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-13f73dae-a858-496b-877f-b1cad455db38 None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:33.285840 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-13f73dae-a858-496b-877f-b1cad455db38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:33.424951 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55fb67-657a-4b12-af3e-b91245012b3e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:33.469932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb301da-a536-474a-8994-db18d06c8bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:33.472165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb301da-a536-474a-8994-db18d06c8bf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:33.473328 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb301da-a536-474a-8994-db18d06c8bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:33.475469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb301da-a536-474a-8994-db18d06c8bf0 admin admin] PATH_INFO: `/v3/projects/c1fd5524ee8f4fa19d6f8dcb4bad3eed` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:33.480098 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb301da-a536-474a-8994-db18d06c8bf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1fd5524ee8f4fa19d6f8dcb4bad3eed)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:33.518469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb301da-a536-474a-8994-db18d06c8bf0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:33.561525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:33.652112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 866/1727] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Sep 7 07:55:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 518 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:33.666509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-edb301da-a536-474a-8994-db18d06c8bf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1fd5524ee8f4fa19d6f8dcb4bad3eed', 'request_id': 'req-edb301da-a536-474a-8994-db18d06c8bf0'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:33.681909 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-edb301da-a536-474a-8994-db18d06c8bf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1fd5524ee8f4fa19d6f8dcb4bad3eed', 'request_id': 'req-edb301da-a536-474a-8994-db18d06c8bf0'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:33.707738 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-edb301da-a536-474a-8994-db18d06c8bf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1fd5524ee8f4fa19d6f8dcb4bad3eed', 'request_id': 'req-edb301da-a536-474a-8994-db18d06c8bf0'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:33.771599 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-edb301da-a536-474a-8994-db18d06c8bf0 admin admin] The token cache is being invalidate because project c1fd5524ee8f4fa19d6f8dcb4bad3eed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:33.771913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-edb301da-a536-474a-8994-db18d06c8bf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:33.780123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-edb301da-a536-474a-8994-db18d06c8bf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:33.790713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 862/1728] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:33 2022] DELETE /identity/v3/projects/c1fd5524ee8f4fa19d6f8dcb4bad3eed => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:33.796599 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efead834-096f-40df-ad11-ed43b68aaa17 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:33.796952 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efead834-096f-40df-ad11-ed43b68aaa17 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:33.797160 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efead834-096f-40df-ad11-ed43b68aaa17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:33.843398 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-efead834-096f-40df-ad11-ed43b68aaa17 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:33.844658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-efead834-096f-40df-ad11-ed43b68aaa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:33.957128 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb301da-a536-474a-8994-db18d06c8bf0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:33.958699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-edb301da-a536-474a-8994-db18d06c8bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:34.078053 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 867/1729] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:34.091230 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c717d87-786d-4e94-9f94-e65264983fa5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:34.091536 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c717d87-786d-4e94-9f94-e65264983fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:34.093632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c717d87-786d-4e94-9f94-e65264983fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:34.262804 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6c717d87-786d-4e94-9f94-e65264983fa5 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:34.266021 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6c717d87-786d-4e94-9f94-e65264983fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:34.311790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 868/1730] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:34.317346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c717d87-786d-4e94-9f94-e65264983fa5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:34.349108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c717d87-786d-4e94-9f94-e65264983fa5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:34.377476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c415f6-6f80-4be1-a419-a1a728697177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:34.378316 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c415f6-6f80-4be1-a419-a1a728697177 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:34.378757 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c415f6-6f80-4be1-a419-a1a728697177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:34.379171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c415f6-6f80-4be1-a419-a1a728697177 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:34.381177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c415f6-6f80-4be1-a419-a1a728697177 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:34.381645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c415f6-6f80-4be1-a419-a1a728697177 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:34.384220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c415f6-6f80-4be1-a419-a1a728697177 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:34.432906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 869/1731] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:34.435618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c415f6-6f80-4be1-a419-a1a728697177 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:34.442982 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb301da-a536-474a-8994-db18d06c8bf0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:34.458330 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121bdac-a849-4326-886e-2e74b998ef44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:34.460316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121bdac-a849-4326-886e-2e74b998ef44 service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:34.460624 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121bdac-a849-4326-886e-2e74b998ef44 service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:34.464238 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121bdac-a849-4326-886e-2e74b998ef44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:34.470276 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c121bdac-a849-4326-886e-2e74b998ef44 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:34.473231 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c415f6-6f80-4be1-a419-a1a728697177 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:34.486122 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095951a-e6fb-4325-9137-6603aadf2c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:34.486745 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095951a-e6fb-4325-9137-6603aadf2c94 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:34.487078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095951a-e6fb-4325-9137-6603aadf2c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:34.487360 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095951a-e6fb-4325-9137-6603aadf2c94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:34.488009 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095951a-e6fb-4325-9137-6603aadf2c94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:34.495387 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095951a-e6fb-4325-9137-6603aadf2c94 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:34.557631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 870/1732] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:34 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:34.559105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095951a-e6fb-4325-9137-6603aadf2c94 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:34.643742 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121bdac-a849-4326-886e-2e74b998ef44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:34.647578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121bdac-a849-4326-886e-2e74b998ef44 service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:34.678799 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 863/1733] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 07:55:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 07:55:34.710573 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:34.728302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f37ed-b278-4b4a-af57-d5ab7a4aa390 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:34.728588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f37ed-b278-4b4a-af57-d5ab7a4aa390 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:34.733116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f37ed-b278-4b4a-af57-d5ab7a4aa390 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:34.734973 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095951a-e6fb-4325-9137-6603aadf2c94 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:34.737867 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 864/1734] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 7 07:55:34 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:55:34.749555 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c0331-0e67-497e-ad34-88a8dce0a3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:34.749935 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c0331-0e67-497e-ad34-88a8dce0a3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:34.750222 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c0331-0e67-497e-ad34-88a8dce0a3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:34.750503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c0331-0e67-497e-ad34-88a8dce0a3e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:34.751153 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c0331-0e67-497e-ad34-88a8dce0a3e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:34.753303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c0331-0e67-497e-ad34-88a8dce0a3e2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:34.765740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d439f85-130f-4cb1-8e0b-e68ab3622593 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:34.766050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d439f85-130f-4cb1-8e0b-e68ab3622593 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:34.766329 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d439f85-130f-4cb1-8e0b-e68ab3622593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:34.798090 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 871/1735] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:34 2022] POST /identity/v3/users => generated 285 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:55:34.801761 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c0331-0e67-497e-ad34-88a8dce0a3e2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:34.837800 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-2d439f85-130f-4cb1-8e0b-e68ab3622593 None None] MFA Rules not processed for user `b16b81ab30294bc4a8c87b6d7c3573f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:34.841737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2d439f85-130f-4cb1-8e0b-e68ab3622593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:34.875198 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c0331-0e67-497e-ad34-88a8dce0a3e2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:34.900809 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5c8d3-4d48-4a24-80e7-21a4f80753ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:34.901228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5c8d3-4d48-4a24-80e7-21a4f80753ad admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:34.913699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5c8d3-4d48-4a24-80e7-21a4f80753ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:34.914013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5c8d3-4d48-4a24-80e7-21a4f80753ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:34.918249 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5c8d3-4d48-4a24-80e7-21a4f80753ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:34.919883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5c8d3-4d48-4a24-80e7-21a4f80753ad admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:34.928255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:34.958563 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 872/1736] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:34.963471 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5c8d3-4d48-4a24-80e7-21a4f80753ad admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:34.990327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5c8d3-4d48-4a24-80e7-21a4f80753ad admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:35.003239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00058570-14cb-4743-aa07-17a4ea0dd222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:35.006059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00058570-14cb-4743-aa07-17a4ea0dd222 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:35.022189 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00058570-14cb-4743-aa07-17a4ea0dd222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:35.022189 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00058570-14cb-4743-aa07-17a4ea0dd222 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:35.022189 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00058570-14cb-4743-aa07-17a4ea0dd222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:35.022189 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00058570-14cb-4743-aa07-17a4ea0dd222 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:35.046225 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 873/1737] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:55:35.048630 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00058570-14cb-4743-aa07-17a4ea0dd222 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:35.076494 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00058570-14cb-4743-aa07-17a4ea0dd222 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:35.090839 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3045eeb-af74-43b4-b331-0edeaac280fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:35.091533 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3045eeb-af74-43b4-b331-0edeaac280fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:35.093046 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3045eeb-af74-43b4-b331-0edeaac280fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:35.093330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3045eeb-af74-43b4-b331-0edeaac280fa admin admin] PATH_INFO: `/v3/projects/21083013fac942b092daa65a43f76ff7/users/a106f03c83804bac9337cfa2db258a5d/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:35.137061 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3045eeb-af74-43b4-b331-0edeaac280fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=21083013fac942b092daa65a43f76ff7, user_id=a106f03c83804bac9337cfa2db258a5d, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:35.142060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3045eeb-af74-43b4-b331-0edeaac280fa admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:35.160332 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 865/1738] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 7 07:55:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:55:35.172634 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 874/1739] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:35 2022] PUT /identity/v3/projects/21083013fac942b092daa65a43f76ff7/users/a106f03c83804bac9337cfa2db258a5d/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:35.173750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338e0ad-33db-4b4c-9e84-8e5eafdadd2c None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:35.174045 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338e0ad-33db-4b4c-9e84-8e5eafdadd2c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:35.174322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338e0ad-33db-4b4c-9e84-8e5eafdadd2c None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:35.175903 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 866/1740] 10.222.0.26 () {54 vars in 961 bytes} [Wed Sep 7 07:55:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 07:55:35.184844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefa4af-1a03-42c1-8f59-416d5e8b0c58 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:35.185134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefa4af-1a03-42c1-8f59-416d5e8b0c58 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:35.185414 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefa4af-1a03-42c1-8f59-416d5e8b0c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:35.186792 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb82686-83ff-4125-b350-e2ecce9b4c80 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:35.187237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb82686-83ff-4125-b350-e2ecce9b4c80 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:35.187674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb82686-83ff-4125-b350-e2ecce9b4c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:35.246671 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-beefa4af-1a03-42c1-8f59-416d5e8b0c58 None None] MFA Rules not processed for user `b16b81ab30294bc4a8c87b6d7c3573f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:35.249545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-beefa4af-1a03-42c1-8f59-416d5e8b0c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:35.250010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-cbb82686-83ff-4125-b350-e2ecce9b4c80 None None] MFA Rules not processed for user `a106f03c83804bac9337cfa2db258a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:35.251697 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cbb82686-83ff-4125-b350-e2ecce9b4c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:35.338179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 875/1741] 10.222.0.26 () {58 vars in 1058 bytes} [Wed Sep 7 07:55:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:35.349365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beefa4af-1a03-42c1-8f59-416d5e8b0c58 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:35.371449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beefa4af-1a03-42c1-8f59-416d5e8b0c58 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:35.395126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f5c75-1bf9-4c8d-a50a-c940ab3581c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:35.395126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f5c75-1bf9-4c8d-a50a-c940ab3581c1 service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:35.395126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f5c75-1bf9-4c8d-a50a-c940ab3581c1 service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:35.395126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f5c75-1bf9-4c8d-a50a-c940ab3581c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:35.410009 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 867/1742] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:35.410224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f5c75-1bf9-4c8d-a50a-c940ab3581c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:35.411297 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb82686-83ff-4125-b350-e2ecce9b4c80 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:35.417802 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f5c75-1bf9-4c8d-a50a-c940ab3581c1 service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:35.437512 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 876/1743] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 07:55:35.505882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb82686-83ff-4125-b350-e2ecce9b4c80 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:35.511882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6abac-7662-4965-9554-27ba204c6996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:35.512718 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6abac-7662-4965-9554-27ba204c6996 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:35.513340 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6abac-7662-4965-9554-27ba204c6996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:35.513887 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6abac-7662-4965-9554-27ba204c6996 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:35.515401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6abac-7662-4965-9554-27ba204c6996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:35.517642 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6abac-7662-4965-9554-27ba204c6996 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:35.552946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 868/1744] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:35 2022] POST /identity/v3/projects => generated 367 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:35.556971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f5c75-1bf9-4c8d-a50a-c940ab3581c1 service placement] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:35.569724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f5c75-1bf9-4c8d-a50a-c940ab3581c1 service placement] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:35.640898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebd7b9-a876-4657-851b-ed5cd0fd9efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:35.641514 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebd7b9-a876-4657-851b-ed5cd0fd9efb admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:35.641798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebd7b9-a876-4657-851b-ed5cd0fd9efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:35.642072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebd7b9-a876-4657-851b-ed5cd0fd9efb admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:35.642782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebd7b9-a876-4657-851b-ed5cd0fd9efb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:35.644770 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebd7b9-a876-4657-851b-ed5cd0fd9efb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:35.678896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 877/1745] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:35 2022] POST /identity/v3/users => generated 290 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:35.681816 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6abac-7662-4965-9554-27ba204c6996 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:35.698802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6abac-7662-4965-9554-27ba204c6996 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:35.706575 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cdaf8-5be7-4157-8cf3-7bb38a571b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:35.708744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cdaf8-5be7-4157-8cf3-7bb38a571b5f admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:35.710105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cdaf8-5be7-4157-8cf3-7bb38a571b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:35.710624 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cdaf8-5be7-4157-8cf3-7bb38a571b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:35.711542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6cdaf8-5be7-4157-8cf3-7bb38a571b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:35.713631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6cdaf8-5be7-4157-8cf3-7bb38a571b5f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:35.743951 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 869/1746] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:35.749045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebd7b9-a876-4657-851b-ed5cd0fd9efb admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:35.775993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebd7b9-a876-4657-851b-ed5cd0fd9efb admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:35.803380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d3bb2-63bc-4f49-83a8-d979c2d952a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:35.804291 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d3bb2-63bc-4f49-83a8-d979c2d952a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:35.805823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d3bb2-63bc-4f49-83a8-d979c2d952a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:35.806231 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d3bb2-63bc-4f49-83a8-d979c2d952a4 admin admin] PATH_INFO: `/v3/projects/e58f65be55ec400e8fdf7bea456ac245/users/8b6158c96bd74edd8647bf7cf6a4f359/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:35.809526 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:35.827715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d3bb2-63bc-4f49-83a8-d979c2d952a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e58f65be55ec400e8fdf7bea456ac245, user_id=8b6158c96bd74edd8647bf7cf6a4f359, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:35.831105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d3bb2-63bc-4f49-83a8-d979c2d952a4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:35.853981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 878/1747] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:35 2022] PUT /identity/v3/projects/e58f65be55ec400e8fdf7bea456ac245/users/8b6158c96bd74edd8647bf7cf6a4f359/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:35.854432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cdaf8-5be7-4157-8cf3-7bb38a571b5f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:35.872833 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cdaf8-5be7-4157-8cf3-7bb38a571b5f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:35.888971 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:35.942711 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640f16f-e38b-40a6-8904-0be127482e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:35.943540 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640f16f-e38b-40a6-8904-0be127482e26 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:35.943756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640f16f-e38b-40a6-8904-0be127482e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:35.944032 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640f16f-e38b-40a6-8904-0be127482e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:35.944727 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2640f16f-e38b-40a6-8904-0be127482e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:35.946886 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2640f16f-e38b-40a6-8904-0be127482e26 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:35.958291 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 870/1748] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:35.962518 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d3bb2-63bc-4f49-83a8-d979c2d952a4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:35.973716 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d3bb2-63bc-4f49-83a8-d979c2d952a4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:35.978987 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef433d-6b32-49ba-b282-092598e40b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:35.979650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef433d-6b32-49ba-b282-092598e40b67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:35.979929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef433d-6b32-49ba-b282-092598e40b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:35.980242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef433d-6b32-49ba-b282-092598e40b67 admin admin] PATH_INFO: `/v3/projects/e58f65be55ec400e8fdf7bea456ac245/users/8b6158c96bd74edd8647bf7cf6a4f359/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:35.982346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef433d-6b32-49ba-b282-092598e40b67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e58f65be55ec400e8fdf7bea456ac245, user_id=8b6158c96bd74edd8647bf7cf6a4f359, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:35.984853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef433d-6b32-49ba-b282-092598e40b67 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:35.997307 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 879/1749] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:35 2022] PUT /identity/v3/projects/e58f65be55ec400e8fdf7bea456ac245/users/8b6158c96bd74edd8647bf7cf6a4f359/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:36.010891 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216daa66-c0d8-4d2c-9bf8-28970dc9fcd0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:36.012208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216daa66-c0d8-4d2c-9bf8-28970dc9fcd0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:36.012705 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216daa66-c0d8-4d2c-9bf8-28970dc9fcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:36.061439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-216daa66-c0d8-4d2c-9bf8-28970dc9fcd0 None None] MFA Rules not processed for user `8b6158c96bd74edd8647bf7cf6a4f359`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:36.063652 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-216daa66-c0d8-4d2c-9bf8-28970dc9fcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:36.165130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 871/1750] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:36.180001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef433d-6b32-49ba-b282-092598e40b67 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:36.182852 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5aef433d-6b32-49ba-b282-092598e40b67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:36.261623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef433d-6b32-49ba-b282-092598e40b67 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:36.266553 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cf46ab-678a-438a-99f6-149bcc5a97b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:36.267288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf46ab-678a-438a-99f6-149bcc5a97b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:36.267676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf46ab-678a-438a-99f6-149bcc5a97b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:36.268056 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf46ab-678a-438a-99f6-149bcc5a97b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:36.278917 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cf46ab-678a-438a-99f6-149bcc5a97b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:36.280552 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cf46ab-678a-438a-99f6-149bcc5a97b0 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:36.291290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 880/1751] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:36.949673 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216daa66-c0d8-4d2c-9bf8-28970dc9fcd0 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:36.951563 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-216daa66-c0d8-4d2c-9bf8-28970dc9fcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:37.148303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a1b23-f071-431b-813e-a1136e367e3b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:37.148881 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a1b23-f071-431b-813e-a1136e367e3b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:37.149637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a1b23-f071-431b-813e-a1136e367e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:37.233005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-066a1b23-f071-431b-813e-a1136e367e3b None None] MFA Rules not processed for user `e07bc40275734f628fcc2f62dd4dd0e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:37.242457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-066a1b23-f071-431b-813e-a1136e367e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:37.277716 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216daa66-c0d8-4d2c-9bf8-28970dc9fcd0 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:37.290447 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0ea8f-830e-45da-954e-ec721ac9755e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:37.291126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0ea8f-830e-45da-954e-ec721ac9755e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:37.291403 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0ea8f-830e-45da-954e-ec721ac9755e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:37.295472 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0ea8f-830e-45da-954e-ec721ac9755e admin admin] PATH_INFO: `/v3/users/fd5d3de8622e4295b18e790d09217a48` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:37.297342 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0ea8f-830e-45da-954e-ec721ac9755e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd5d3de8622e4295b18e790d09217a48)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:37.320054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0ea8f-830e-45da-954e-ec721ac9755e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:37.442879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:37.455754 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6cd0ea8f-830e-45da-954e-ec721ac9755e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd5d3de8622e4295b18e790d09217a48'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:37.455754 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6cd0ea8f-830e-45da-954e-ec721ac9755e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd5d3de8622e4295b18e790d09217a48'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:37.500537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6cd0ea8f-830e-45da-954e-ec721ac9755e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd5d3de8622e4295b18e790d09217a48'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:37.503315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6cd0ea8f-830e-45da-954e-ec721ac9755e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd5d3de8622e4295b18e790d09217a48'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:37.527152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6cd0ea8f-830e-45da-954e-ec721ac9755e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd5d3de8622e4295b18e790d09217a48'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:37.553828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 881/1752] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:37.553828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccfa59-c721-45cb-b737-fbc310251a01 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:37.553828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccfa59-c721-45cb-b737-fbc310251a01 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:37.554273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccfa59-c721-45cb-b737-fbc310251a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:37.572557 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 872/1753] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:36 2022] DELETE /identity/v3/users/fd5d3de8622e4295b18e790d09217a48 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:37.593909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-40ccfa59-c721-45cb-b737-fbc310251a01 None None] MFA Rules not processed for user `330a38822d33439191773e9a8b4cbe04`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:37.595290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-40ccfa59-c721-45cb-b737-fbc310251a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:37.802724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 882/1754] 10.222.0.27 () {58 vars in 1015 bytes} [Wed Sep 7 07:55:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:37.926352 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0ea8f-830e-45da-954e-ec721ac9755e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:37.929521 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6cd0ea8f-830e-45da-954e-ec721ac9755e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:37.962281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0ea8f-830e-45da-954e-ec721ac9755e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:37.971809 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000f0bc-9eeb-464b-8a65-01f382a9e0a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:37.973260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000f0bc-9eeb-464b-8a65-01f382a9e0a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:37.973499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000f0bc-9eeb-464b-8a65-01f382a9e0a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:37.973826 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000f0bc-9eeb-464b-8a65-01f382a9e0a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:37.995036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000f0bc-9eeb-464b-8a65-01f382a9e0a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:37.998685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000f0bc-9eeb-464b-8a65-01f382a9e0a0 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:38.046866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 873/1755] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:38.485994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccfa59-c721-45cb-b737-fbc310251a01 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:38.487661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-40ccfa59-c721-45cb-b737-fbc310251a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:38.615026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b70d72-3966-4e74-b3df-9053b2a46cbc None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:38.615194 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b70d72-3966-4e74-b3df-9053b2a46cbc None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:38.615624 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b70d72-3966-4e74-b3df-9053b2a46cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:38.726114 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:38.730117 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c0b70d72-3966-4e74-b3df-9053b2a46cbc None None] MFA Rules not processed for user `457d69a7306d46cd8445f887eb95c517`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:38.741777 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c0b70d72-3966-4e74-b3df-9053b2a46cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:38.893447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccfa59-c721-45cb-b737-fbc310251a01 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:38.926961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e6be8-516f-4c5c-b568-aeece939099f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:38.934160 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e6be8-516f-4c5c-b568-aeece939099f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:38.941181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e6be8-516f-4c5c-b568-aeece939099f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:38.941891 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e6be8-516f-4c5c-b568-aeece939099f admin admin] PATH_INFO: `/v3/projects/8a5b8f3bfd45460993e41576399a033e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:38.945298 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e6be8-516f-4c5c-b568-aeece939099f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a5b8f3bfd45460993e41576399a033e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:38.953227 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e6be8-516f-4c5c-b568-aeece939099f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:39.038334 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 874/1756] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:39.089507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-371e6be8-516f-4c5c-b568-aeece939099f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a5b8f3bfd45460993e41576399a033e', 'request_id': 'req-371e6be8-516f-4c5c-b568-aeece939099f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:39.098526 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-371e6be8-516f-4c5c-b568-aeece939099f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a5b8f3bfd45460993e41576399a033e', 'request_id': 'req-371e6be8-516f-4c5c-b568-aeece939099f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:39.107730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-371e6be8-516f-4c5c-b568-aeece939099f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a5b8f3bfd45460993e41576399a033e', 'request_id': 'req-371e6be8-516f-4c5c-b568-aeece939099f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:39.128478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-371e6be8-516f-4c5c-b568-aeece939099f admin admin] The token cache is being invalidate because project 8a5b8f3bfd45460993e41576399a033e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:39.129258 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-371e6be8-516f-4c5c-b568-aeece939099f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:39.130231 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-371e6be8-516f-4c5c-b568-aeece939099f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:39.132809 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 883/1757] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:38 2022] DELETE /identity/v3/projects/8a5b8f3bfd45460993e41576399a033e => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:39.137889 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b70d72-3966-4e74-b3df-9053b2a46cbc None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:39.161828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b70d72-3966-4e74-b3df-9053b2a46cbc None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:39.412203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e8038-8902-4463-bdc8-eadab2dcceac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:39.413535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e8038-8902-4463-bdc8-eadab2dcceac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:39.413657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e8038-8902-4463-bdc8-eadab2dcceac admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:39.418409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e8038-8902-4463-bdc8-eadab2dcceac admin admin] PATH_INFO: `/v3/users/299b501213f5480992bc8f57e049573b` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:39.418409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e8038-8902-4463-bdc8-eadab2dcceac admin admin] RBAC: Authorizing `identity:delete_user(user_id=299b501213f5480992bc8f57e049573b)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:39.418409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e8038-8902-4463-bdc8-eadab2dcceac admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:39.580118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f72e8038-8902-4463-bdc8-eadab2dcceac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '299b501213f5480992bc8f57e049573b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:39.581007 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f72e8038-8902-4463-bdc8-eadab2dcceac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '299b501213f5480992bc8f57e049573b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:39.596012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:39.628427 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f72e8038-8902-4463-bdc8-eadab2dcceac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '299b501213f5480992bc8f57e049573b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:39.630838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f72e8038-8902-4463-bdc8-eadab2dcceac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '299b501213f5480992bc8f57e049573b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:39.665669 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f72e8038-8902-4463-bdc8-eadab2dcceac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '299b501213f5480992bc8f57e049573b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:39.701769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 875/1758] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:39 2022] DELETE /identity/v3/users/299b501213f5480992bc8f57e049573b => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:40.965697 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e6be8-516f-4c5c-b568-aeece939099f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:40.968912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-371e6be8-516f-4c5c-b568-aeece939099f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:40.995364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e6be8-516f-4c5c-b568-aeece939099f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:41.160320 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337b572-1361-4b59-8ff2-b895f6d511e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:41.161375 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337b572-1361-4b59-8ff2-b895f6d511e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:41.161899 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337b572-1361-4b59-8ff2-b895f6d511e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:41.162515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337b572-1361-4b59-8ff2-b895f6d511e3 admin admin] PATH_INFO: `/v3/users/a106f03c83804bac9337cfa2db258a5d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:41.165344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337b572-1361-4b59-8ff2-b895f6d511e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a106f03c83804bac9337cfa2db258a5d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:41.168016 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337b572-1361-4b59-8ff2-b895f6d511e3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:41.275607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f337b572-1361-4b59-8ff2-b895f6d511e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a106f03c83804bac9337cfa2db258a5d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:41.292834 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:41.310417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f337b572-1361-4b59-8ff2-b895f6d511e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a106f03c83804bac9337cfa2db258a5d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:41.349848 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f337b572-1361-4b59-8ff2-b895f6d511e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a106f03c83804bac9337cfa2db258a5d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:41.350585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f337b572-1361-4b59-8ff2-b895f6d511e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a106f03c83804bac9337cfa2db258a5d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:41.370893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f337b572-1361-4b59-8ff2-b895f6d511e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a106f03c83804bac9337cfa2db258a5d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:41.392310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 884/1759] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:40 2022] DELETE /identity/v3/users/a106f03c83804bac9337cfa2db258a5d => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:41.394061 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf76fa4-8262-4ec7-b8a4-0cdcad8601d9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:41.394561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf76fa4-8262-4ec7-b8a4-0cdcad8601d9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:41.394648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf76fa4-8262-4ec7-b8a4-0cdcad8601d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:41.461071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-adf76fa4-8262-4ec7-b8a4-0cdcad8601d9 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:41.465620 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-adf76fa4-8262-4ec7-b8a4-0cdcad8601d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:41.608842 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337b572-1361-4b59-8ff2-b895f6d511e3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:41.610974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f337b572-1361-4b59-8ff2-b895f6d511e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:41.659702 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 876/1760] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:41.703842 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf76fa4-8262-4ec7-b8a4-0cdcad8601d9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:41.705564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-adf76fa4-8262-4ec7-b8a4-0cdcad8601d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:41.888833 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf76fa4-8262-4ec7-b8a4-0cdcad8601d9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:41.902874 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fda42-34a8-4480-8367-c80e3b4c01f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:41.903700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fda42-34a8-4480-8367-c80e3b4c01f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:41.904166 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fda42-34a8-4480-8367-c80e3b4c01f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:41.904632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fda42-34a8-4480-8367-c80e3b4c01f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:41.912372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337b572-1361-4b59-8ff2-b895f6d511e3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:41.928918 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fda42-34a8-4480-8367-c80e3b4c01f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:41.931294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fda42-34a8-4480-8367-c80e3b4c01f0 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:41.931858 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13727817-156f-4bdb-8c57-3e7d9e81e560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:41.932617 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13727817-156f-4bdb-8c57-3e7d9e81e560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:41.932891 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13727817-156f-4bdb-8c57-3e7d9e81e560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:41.933278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13727817-156f-4bdb-8c57-3e7d9e81e560 admin admin] PATH_INFO: `/v3/projects/644c9956bae842c7a6f0ce40d2a94591` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:41.934524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13727817-156f-4bdb-8c57-3e7d9e81e560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=644c9956bae842c7a6f0ce40d2a94591)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:41.936854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13727817-156f-4bdb-8c57-3e7d9e81e560 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:41.958222 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 877/1761] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:41.959517 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fda42-34a8-4480-8367-c80e3b4c01f0 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:41.963811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-680fda42-34a8-4480-8367-c80e3b4c01f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:42.002561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fda42-34a8-4480-8367-c80e3b4c01f0 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:42.021897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d0510-5afd-4cc9-b451-18d1b4ce79ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:42.025157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d0510-5afd-4cc9-b451-18d1b4ce79ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:42.025540 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d0510-5afd-4cc9-b451-18d1b4ce79ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:42.031161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d0510-5afd-4cc9-b451-18d1b4ce79ba admin admin] PATH_INFO: `/v3/users/3841bac8b11e4f70ade14c7e9132793c` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:42.031614 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d0510-5afd-4cc9-b451-18d1b4ce79ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=3841bac8b11e4f70ade14c7e9132793c)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:42.037548 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d0510-5afd-4cc9-b451-18d1b4ce79ba admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:42.059314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-13727817-156f-4bdb-8c57-3e7d9e81e560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '644c9956bae842c7a6f0ce40d2a94591', 'request_id': 'req-13727817-156f-4bdb-8c57-3e7d9e81e560'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:42.076755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-13727817-156f-4bdb-8c57-3e7d9e81e560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '644c9956bae842c7a6f0ce40d2a94591', 'request_id': 'req-13727817-156f-4bdb-8c57-3e7d9e81e560'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:42.084937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-13727817-156f-4bdb-8c57-3e7d9e81e560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '644c9956bae842c7a6f0ce40d2a94591', 'request_id': 'req-13727817-156f-4bdb-8c57-3e7d9e81e560'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:42.113981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-13727817-156f-4bdb-8c57-3e7d9e81e560 admin admin] The token cache is being invalidate because project 644c9956bae842c7a6f0ce40d2a94591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:42.114573 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-13727817-156f-4bdb-8c57-3e7d9e81e560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:42.117222 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-13727817-156f-4bdb-8c57-3e7d9e81e560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:42.121284 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 885/1762] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:41 2022] DELETE /identity/v3/projects/644c9956bae842c7a6f0ce40d2a94591 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:42.129592 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b51fd-ea21-4470-80b7-04493cd76bcc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:42.130906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b51fd-ea21-4470-80b7-04493cd76bcc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:42.133207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b51fd-ea21-4470-80b7-04493cd76bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:42.164298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-553d0510-5afd-4cc9-b451-18d1b4ce79ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3841bac8b11e4f70ade14c7e9132793c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:42.166268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-553d0510-5afd-4cc9-b451-18d1b4ce79ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3841bac8b11e4f70ade14c7e9132793c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:42.227477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-553d0510-5afd-4cc9-b451-18d1b4ce79ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3841bac8b11e4f70ade14c7e9132793c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:42.230090 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-081b51fd-ea21-4470-80b7-04493cd76bcc None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:42.230747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-553d0510-5afd-4cc9-b451-18d1b4ce79ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3841bac8b11e4f70ade14c7e9132793c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:42.233470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:42.237204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-081b51fd-ea21-4470-80b7-04493cd76bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:42.254283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-553d0510-5afd-4cc9-b451-18d1b4ce79ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3841bac8b11e4f70ade14c7e9132793c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:42.272168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 878/1763] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:41 2022] DELETE /identity/v3/users/3841bac8b11e4f70ade14c7e9132793c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:42.278306 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b5ef5-441e-468d-9800-38d61e54e4ed None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:42.278718 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b5ef5-441e-468d-9800-38d61e54e4ed None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:42.279309 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b5ef5-441e-468d-9800-38d61e54e4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:42.362285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-911b5ef5-441e-468d-9800-38d61e54e4ed None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:42.365130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-911b5ef5-441e-468d-9800-38d61e54e4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:42.476698 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 886/1764] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:42.484331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33928f-1928-4f48-acc8-a9dd00583526 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:42.484617 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33928f-1928-4f48-acc8-a9dd00583526 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:42.485210 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33928f-1928-4f48-acc8-a9dd00583526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:42.494476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 879/1765] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:42.524531 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:42.525328 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b5ef5-441e-468d-9800-38d61e54e4ed None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:42.537254 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-911b5ef5-441e-468d-9800-38d61e54e4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:42.542597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9c33928f-1928-4f48-acc8-a9dd00583526 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:42.544232 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9c33928f-1928-4f48-acc8-a9dd00583526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:42.553382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 887/1766] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:42.568607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33928f-1928-4f48-acc8-a9dd00583526 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:42.595979 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33928f-1928-4f48-acc8-a9dd00583526 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:42.609730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff955df2-ffe7-4d9c-93a6-edeafe1fc634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:42.611961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff955df2-ffe7-4d9c-93a6-edeafe1fc634 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:42.612591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff955df2-ffe7-4d9c-93a6-edeafe1fc634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:42.613328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff955df2-ffe7-4d9c-93a6-edeafe1fc634 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:42.614730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff955df2-ffe7-4d9c-93a6-edeafe1fc634 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:42.615344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff955df2-ffe7-4d9c-93a6-edeafe1fc634 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:42.621454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff955df2-ffe7-4d9c-93a6-edeafe1fc634 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:42.665854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 888/1767] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:42.668127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff955df2-ffe7-4d9c-93a6-edeafe1fc634 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:42.679210 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b5ef5-441e-468d-9800-38d61e54e4ed None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:42.682277 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff955df2-ffe7-4d9c-93a6-edeafe1fc634 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:42.684016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e97f3-4508-4d4f-9b69-e3cdad79a53f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:42.685050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e97f3-4508-4d4f-9b69-e3cdad79a53f service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:42.685708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e97f3-4508-4d4f-9b69-e3cdad79a53f service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:42.686134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e97f3-4508-4d4f-9b69-e3cdad79a53f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:42.688604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409dab6-2a60-4df6-acf7-94beff217117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:42.689428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409dab6-2a60-4df6-acf7-94beff217117 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:42.689909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409dab6-2a60-4df6-acf7-94beff217117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:42.690421 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409dab6-2a60-4df6-acf7-94beff217117 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:42.691157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409dab6-2a60-4df6-acf7-94beff217117 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:42.693620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409dab6-2a60-4df6-acf7-94beff217117 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:42.699009 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e97f3-4508-4d4f-9b69-e3cdad79a53f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:42.700984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e97f3-4508-4d4f-9b69-e3cdad79a53f service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:42.722682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 880/1768] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:42.762501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e97f3-4508-4d4f-9b69-e3cdad79a53f service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:42.768353 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 889/1769] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:42 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:42.781915 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e97f3-4508-4d4f-9b69-e3cdad79a53f service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:42.924659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544dc79-4da1-48ec-a628-963fa3b086ef None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:42.925160 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544dc79-4da1-48ec-a628-963fa3b086ef None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:42.925640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544dc79-4da1-48ec-a628-963fa3b086ef None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:42.927449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 890/1770] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 7 07:55:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:55:42.964908 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22755a68-a6ca-45e9-af57-15519339a490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:42.965434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22755a68-a6ca-45e9-af57-15519339a490 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:42.966828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22755a68-a6ca-45e9-af57-15519339a490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:42.968187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22755a68-a6ca-45e9-af57-15519339a490 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:42.968831 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22755a68-a6ca-45e9-af57-15519339a490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:42.975256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22755a68-a6ca-45e9-af57-15519339a490 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:42.979569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba843f19-455b-4271-a95e-0a875f16c608 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:42.980158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba843f19-455b-4271-a95e-0a875f16c608 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:42.980600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba843f19-455b-4271-a95e-0a875f16c608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:43.031956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ba843f19-455b-4271-a95e-0a875f16c608 None None] MFA Rules not processed for user `81d5689e9a344bafb74ca423ffd2b42a`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:43.035609 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ba843f19-455b-4271-a95e-0a875f16c608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:43.062042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 881/1771] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:42 2022] POST /identity/v3/users => generated 294 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:43.063074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22755a68-a6ca-45e9-af57-15519339a490 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:43.088611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22755a68-a6ca-45e9-af57-15519339a490 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:43.100627 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa51d0a-3270-4c94-8406-b5f7bdf2007a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:43.101513 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa51d0a-3270-4c94-8406-b5f7bdf2007a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:43.102824 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa51d0a-3270-4c94-8406-b5f7bdf2007a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:43.103200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa51d0a-3270-4c94-8406-b5f7bdf2007a admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:43.103876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa51d0a-3270-4c94-8406-b5f7bdf2007a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:43.107550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa51d0a-3270-4c94-8406-b5f7bdf2007a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:43.145598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 882/1772] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:43.148241 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa51d0a-3270-4c94-8406-b5f7bdf2007a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:43.177537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa51d0a-3270-4c94-8406-b5f7bdf2007a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:43.191104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61988a05-f5e5-498b-928f-cc394e2e3ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:43.191912 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61988a05-f5e5-498b-928f-cc394e2e3ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:43.192273 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61988a05-f5e5-498b-928f-cc394e2e3ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:43.192697 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61988a05-f5e5-498b-928f-cc394e2e3ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:43.198139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61988a05-f5e5-498b-928f-cc394e2e3ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:43.207211 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61988a05-f5e5-498b-928f-cc394e2e3ee5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:43.209434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:43.251492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 883/1773] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:43.252546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61988a05-f5e5-498b-928f-cc394e2e3ee5 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:43.308572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 891/1774] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 7 07:55:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:43.309163 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61988a05-f5e5-498b-928f-cc394e2e3ee5 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:43.320140 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28958f-040d-4cbe-8890-c9650241569b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:43.322853 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28958f-040d-4cbe-8890-c9650241569b admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:43.322989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28958f-040d-4cbe-8890-c9650241569b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:43.325098 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28958f-040d-4cbe-8890-c9650241569b admin admin] PATH_INFO: `/v3/projects/7796d628744b425893d3408c347f2b60/users/b5088b0e95354eedb311271b2bd2710a/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:43.402732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf28958f-040d-4cbe-8890-c9650241569b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7796d628744b425893d3408c347f2b60, user_id=b5088b0e95354eedb311271b2bd2710a, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:43.406332 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf28958f-040d-4cbe-8890-c9650241569b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:43.449843 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 884/1775] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:43 2022] PUT /identity/v3/projects/7796d628744b425893d3408c347f2b60/users/b5088b0e95354eedb311271b2bd2710a/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:43.464546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec3798-bd21-4815-80da-5571cdee502f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:43.464850 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec3798-bd21-4815-80da-5571cdee502f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:43.465126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec3798-bd21-4815-80da-5571cdee502f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:43.563525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-02ec3798-bd21-4815-80da-5571cdee502f None None] MFA Rules not processed for user `b5088b0e95354eedb311271b2bd2710a`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:43.565166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-02ec3798-bd21-4815-80da-5571cdee502f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:43.732183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28958f-040d-4cbe-8890-c9650241569b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:43.732183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-bf28958f-040d-4cbe-8890-c9650241569b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:43.807583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 892/1776] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:43.828812 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec3798-bd21-4815-80da-5571cdee502f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:43.998399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:44.032990 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec3798-bd21-4815-80da-5571cdee502f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:44.041618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d37a5-0bc7-41be-9e74-8701d625562d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:44.042291 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d37a5-0bc7-41be-9e74-8701d625562d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:44.042729 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d37a5-0bc7-41be-9e74-8701d625562d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:44.043130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d37a5-0bc7-41be-9e74-8701d625562d admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:44.043895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d37a5-0bc7-41be-9e74-8701d625562d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:44.047058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d37a5-0bc7-41be-9e74-8701d625562d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:44.100099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28958f-040d-4cbe-8890-c9650241569b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:44.111667 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 893/1777] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:43 2022] POST /identity/v3/projects => generated 385 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:44.116131 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d37a5-0bc7-41be-9e74-8701d625562d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:44.126834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc20172-0f29-4c4c-bad6-90335893ed00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:44.128767 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc20172-0f29-4c4c-bad6-90335893ed00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:44.129931 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc20172-0f29-4c4c-bad6-90335893ed00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:44.132486 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc20172-0f29-4c4c-bad6-90335893ed00 admin admin] PATH_INFO: `/v3/projects/21083013fac942b092daa65a43f76ff7` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:44.142584 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc20172-0f29-4c4c-bad6-90335893ed00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21083013fac942b092daa65a43f76ff7)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:44.146139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc20172-0f29-4c4c-bad6-90335893ed00 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:44.291634 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6fc20172-0f29-4c4c-bad6-90335893ed00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21083013fac942b092daa65a43f76ff7', 'request_id': 'req-6fc20172-0f29-4c4c-bad6-90335893ed00'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:44.307008 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6fc20172-0f29-4c4c-bad6-90335893ed00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21083013fac942b092daa65a43f76ff7', 'request_id': 'req-6fc20172-0f29-4c4c-bad6-90335893ed00'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:44.316950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d37a5-0bc7-41be-9e74-8701d625562d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:44.319170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6fc20172-0f29-4c4c-bad6-90335893ed00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21083013fac942b092daa65a43f76ff7', 'request_id': 'req-6fc20172-0f29-4c4c-bad6-90335893ed00'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:44.353322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6fc20172-0f29-4c4c-bad6-90335893ed00 admin admin] The token cache is being invalidate because project 21083013fac942b092daa65a43f76ff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:44.355300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6fc20172-0f29-4c4c-bad6-90335893ed00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:44.357707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6fc20172-0f29-4c4c-bad6-90335893ed00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:44.365299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 885/1778] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:43 2022] DELETE /identity/v3/projects/21083013fac942b092daa65a43f76ff7 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:44.369914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc20172-0f29-4c4c-bad6-90335893ed00 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:44.371806 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6fc20172-0f29-4c4c-bad6-90335893ed00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:44.531689 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f85b83-fd62-4216-8e0e-f05e8172346c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:44.532489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f85b83-fd62-4216-8e0e-f05e8172346c admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:44.532702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f85b83-fd62-4216-8e0e-f05e8172346c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:44.537873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f85b83-fd62-4216-8e0e-f05e8172346c admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:44.544296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f85b83-fd62-4216-8e0e-f05e8172346c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:44.552641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f85b83-fd62-4216-8e0e-f05e8172346c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:44.623319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 894/1779] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:44 2022] POST /identity/v3/users => generated 299 bytes in 507 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:44.626450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f85b83-fd62-4216-8e0e-f05e8172346c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:44.635916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc20172-0f29-4c4c-bad6-90335893ed00 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:44.636216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-06f85b83-fd62-4216-8e0e-f05e8172346c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:44.641848 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8290ab-70c6-4194-8743-4db2a12b60d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:44.642511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8290ab-70c6-4194-8743-4db2a12b60d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:44.642793 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8290ab-70c6-4194-8743-4db2a12b60d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:44.643069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8290ab-70c6-4194-8743-4db2a12b60d7 admin admin] PATH_INFO: `/v3/users/8b6158c96bd74edd8647bf7cf6a4f359` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:44.644377 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8290ab-70c6-4194-8743-4db2a12b60d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b6158c96bd74edd8647bf7cf6a4f359)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:44.646378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8290ab-70c6-4194-8743-4db2a12b60d7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:44.650368 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:44.703454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f85b83-fd62-4216-8e0e-f05e8172346c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:44.712299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7655e3-6720-4051-a4f4-676cf8d28453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:44.713382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7655e3-6720-4051-a4f4-676cf8d28453 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:44.713779 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7655e3-6720-4051-a4f4-676cf8d28453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:44.714044 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7655e3-6720-4051-a4f4-676cf8d28453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:44.714858 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7655e3-6720-4051-a4f4-676cf8d28453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:44.716662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7655e3-6720-4051-a4f4-676cf8d28453 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:44.737106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5c8290ab-70c6-4194-8743-4db2a12b60d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b6158c96bd74edd8647bf7cf6a4f359'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:44.744865 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5c8290ab-70c6-4194-8743-4db2a12b60d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b6158c96bd74edd8647bf7cf6a4f359'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:44.761341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 895/1780] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:44.762376 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7655e3-6720-4051-a4f4-676cf8d28453 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:44.768680 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7b7655e3-6720-4051-a4f4-676cf8d28453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:44.774337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5c8290ab-70c6-4194-8743-4db2a12b60d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b6158c96bd74edd8647bf7cf6a4f359'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:44.775328 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5c8290ab-70c6-4194-8743-4db2a12b60d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b6158c96bd74edd8647bf7cf6a4f359'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:44.793169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5c8290ab-70c6-4194-8743-4db2a12b60d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b6158c96bd74edd8647bf7cf6a4f359'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:44.818878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 886/1781] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:44 2022] DELETE /identity/v3/users/8b6158c96bd74edd8647bf7cf6a4f359 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:44.820096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8290ab-70c6-4194-8743-4db2a12b60d7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:44.824762 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5c8290ab-70c6-4194-8743-4db2a12b60d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:44.963141 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:45.006542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8290ab-70c6-4194-8743-4db2a12b60d7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:45.020704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0e481-8cb9-4223-9984-146a264cce6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:45.022724 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0e481-8cb9-4223-9984-146a264cce6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:45.023023 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0e481-8cb9-4223-9984-146a264cce6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:45.025006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0e481-8cb9-4223-9984-146a264cce6b admin admin] PATH_INFO: `/v3/projects/bb486dfc42a34848ba2fef9ee293ae0c/users/bc687840a22c428ba5df20a5d9703ec5/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:45.060440 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7655e3-6720-4051-a4f4-676cf8d28453 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:45.075541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0e481-8cb9-4223-9984-146a264cce6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb486dfc42a34848ba2fef9ee293ae0c, user_id=bc687840a22c428ba5df20a5d9703ec5, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:45.081226 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0e481-8cb9-4223-9984-146a264cce6b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:45.131355 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 887/1782] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:44 2022] PUT /identity/v3/projects/bb486dfc42a34848ba2fef9ee293ae0c/users/bc687840a22c428ba5df20a5d9703ec5/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:45.140611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0e481-8cb9-4223-9984-146a264cce6b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:45.214900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45549be7-b7b1-47fe-9262-6c900b45545b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:45.215728 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45549be7-b7b1-47fe-9262-6c900b45545b service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:45.216348 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45549be7-b7b1-47fe-9262-6c900b45545b service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:45.216941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45549be7-b7b1-47fe-9262-6c900b45545b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:45.220614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-45549be7-b7b1-47fe-9262-6c900b45545b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:45.249665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45549be7-b7b1-47fe-9262-6c900b45545b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:45.251732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45549be7-b7b1-47fe-9262-6c900b45545b service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:45.290681 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 896/1783] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:45.423491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0e481-8cb9-4223-9984-146a264cce6b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:45.430383 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c05ad8-fbce-4712-a17a-92a20f690189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:45.431554 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c05ad8-fbce-4712-a17a-92a20f690189 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:45.432270 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c05ad8-fbce-4712-a17a-92a20f690189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:45.432621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c05ad8-fbce-4712-a17a-92a20f690189 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:45.433409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c05ad8-fbce-4712-a17a-92a20f690189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:45.436447 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c05ad8-fbce-4712-a17a-92a20f690189 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:45.475026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:45.485503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 888/1784] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:45.489724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45549be7-b7b1-47fe-9262-6c900b45545b service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:45.513978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45549be7-b7b1-47fe-9262-6c900b45545b service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:45.523260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7b070-a633-4520-87b7-55cddaf46f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:45.534590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7b070-a633-4520-87b7-55cddaf46f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:45.535313 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7b070-a633-4520-87b7-55cddaf46f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:45.535313 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7b070-a633-4520-87b7-55cddaf46f97 admin admin] PATH_INFO: `/v3/projects/bb486dfc42a34848ba2fef9ee293ae0c/users/bc687840a22c428ba5df20a5d9703ec5/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:45.540474 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7b070-a633-4520-87b7-55cddaf46f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb486dfc42a34848ba2fef9ee293ae0c, user_id=bc687840a22c428ba5df20a5d9703ec5, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:45.551030 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7b070-a633-4520-87b7-55cddaf46f97 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:45.581517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 897/1785] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:45 2022] PUT /identity/v3/projects/bb486dfc42a34848ba2fef9ee293ae0c/users/bc687840a22c428ba5df20a5d9703ec5/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:45.593598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81477fc-2483-4b65-88ef-caebbfaf0110 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:45.593598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81477fc-2483-4b65-88ef-caebbfaf0110 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:45.594034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81477fc-2483-4b65-88ef-caebbfaf0110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:45.600996 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7b070-a633-4520-87b7-55cddaf46f97 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:45.609694 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c3c7b070-a633-4520-87b7-55cddaf46f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:45.739091 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b81477fc-2483-4b65-88ef-caebbfaf0110 None None] MFA Rules not processed for user `bc687840a22c428ba5df20a5d9703ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:45.741688 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b81477fc-2483-4b65-88ef-caebbfaf0110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:45.954432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7b070-a633-4520-87b7-55cddaf46f97 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:45.965800 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:45.966488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:45.966799 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:45.967129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0 admin admin] PATH_INFO: `/v3/projects/c61fd96b94b8426a803325ed9257fc4d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:45.979932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c61fd96b94b8426a803325ed9257fc4d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:45.983276 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:46.030016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 889/1786] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:46.045186 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81477fc-2483-4b65-88ef-caebbfaf0110 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:46.050052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b81477fc-2483-4b65-88ef-caebbfaf0110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:46.106583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c61fd96b94b8426a803325ed9257fc4d', 'request_id': 'req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:46.125777 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c61fd96b94b8426a803325ed9257fc4d', 'request_id': 'req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:46.165382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c61fd96b94b8426a803325ed9257fc4d', 'request_id': 'req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:46.197024 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:46.220182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0 admin admin] The token cache is being invalidate because project c61fd96b94b8426a803325ed9257fc4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:46.221747 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:46.223381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:46.236567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 898/1787] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:45 2022] DELETE /identity/v3/projects/c61fd96b94b8426a803325ed9257fc4d => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:46.239144 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:46.242367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:46.281904 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81477fc-2483-4b65-88ef-caebbfaf0110 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:46.400370 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80bb79-dbd3-4c9d-a75f-6c0066b789e0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:46.416443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67913401-f7bd-4475-b973-dd2da07d3f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:46.418150 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67913401-f7bd-4475-b973-dd2da07d3f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:46.419256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67913401-f7bd-4475-b973-dd2da07d3f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:46.419721 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67913401-f7bd-4475-b973-dd2da07d3f6f admin admin] PATH_INFO: `/v3/users/3fa4c6e147f24fa298c09626ecd100ae` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:46.436606 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67913401-f7bd-4475-b973-dd2da07d3f6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fa4c6e147f24fa298c09626ecd100ae)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:46.463486 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67913401-f7bd-4475-b973-dd2da07d3f6f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:46.466587 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4e380-427f-4761-b1eb-e3b576ac957e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:46.467803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4e380-427f-4761-b1eb-e3b576ac957e service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:46.468178 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4e380-427f-4761-b1eb-e3b576ac957e service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:46.468587 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4e380-427f-4761-b1eb-e3b576ac957e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:46.473977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-13d4e380-427f-4761-b1eb-e3b576ac957e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:46.580004 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-67913401-f7bd-4475-b973-dd2da07d3f6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fa4c6e147f24fa298c09626ecd100ae'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:46.630432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-67913401-f7bd-4475-b973-dd2da07d3f6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fa4c6e147f24fa298c09626ecd100ae'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:46.645807 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-67913401-f7bd-4475-b973-dd2da07d3f6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fa4c6e147f24fa298c09626ecd100ae'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:46.646938 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-67913401-f7bd-4475-b973-dd2da07d3f6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fa4c6e147f24fa298c09626ecd100ae'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:46.655062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:46.663231 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-67913401-f7bd-4475-b973-dd2da07d3f6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fa4c6e147f24fa298c09626ecd100ae'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:46.672939 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 899/1788] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:46 2022] DELETE /identity/v3/users/3fa4c6e147f24fa298c09626ecd100ae => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:46.690436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4e380-427f-4761-b1eb-e3b576ac957e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:46.693006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4e380-427f-4761-b1eb-e3b576ac957e service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:46.706374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 890/1789] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:46.825327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67913401-f7bd-4475-b973-dd2da07d3f6f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:46.826777 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-67913401-f7bd-4475-b973-dd2da07d3f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:46.969883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67913401-f7bd-4475-b973-dd2da07d3f6f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:46.983653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f174811-9484-437c-87e4-42793401bfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:46.984342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f174811-9484-437c-87e4-42793401bfd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:46.984604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f174811-9484-437c-87e4-42793401bfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:46.984874 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f174811-9484-437c-87e4-42793401bfd4 admin admin] PATH_INFO: `/v3/users/b5088b0e95354eedb311271b2bd2710a` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:46.988984 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f174811-9484-437c-87e4-42793401bfd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5088b0e95354eedb311271b2bd2710a)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:46.990834 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f174811-9484-437c-87e4-42793401bfd4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:47.063451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:47.111372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1f174811-9484-437c-87e4-42793401bfd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5088b0e95354eedb311271b2bd2710a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:47.136973 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1f174811-9484-437c-87e4-42793401bfd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5088b0e95354eedb311271b2bd2710a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:47.147917 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1f174811-9484-437c-87e4-42793401bfd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5088b0e95354eedb311271b2bd2710a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:47.148941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1f174811-9484-437c-87e4-42793401bfd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5088b0e95354eedb311271b2bd2710a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:47.163569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1f174811-9484-437c-87e4-42793401bfd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5088b0e95354eedb311271b2bd2710a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:47.176471 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 900/1790] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:46 2022] DELETE /identity/v3/users/b5088b0e95354eedb311271b2bd2710a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:47.177171 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b6316-623d-406d-965a-43af0c9b1508 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:47.177645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b6316-623d-406d-965a-43af0c9b1508 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:47.177718 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b6316-623d-406d-965a-43af0c9b1508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:47.198346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-bb7b6316-623d-406d-965a-43af0c9b1508 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:47.199566 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-bb7b6316-623d-406d-965a-43af0c9b1508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:47.268677 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 891/1791] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:47.322514 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f174811-9484-437c-87e4-42793401bfd4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:47.326129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1f174811-9484-437c-87e4-42793401bfd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:47.374694 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b6316-623d-406d-965a-43af0c9b1508 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:47.377162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-bb7b6316-623d-406d-965a-43af0c9b1508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:47.427112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b6316-623d-406d-965a-43af0c9b1508 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:47.437882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd350665-cbb2-4abf-953b-cf97e421cf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:47.439527 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd350665-cbb2-4abf-953b-cf97e421cf56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:47.440016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd350665-cbb2-4abf-953b-cf97e421cf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:47.440342 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd350665-cbb2-4abf-953b-cf97e421cf56 admin admin] PATH_INFO: `/v3/projects/e58f65be55ec400e8fdf7bea456ac245` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:47.441239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f174811-9484-437c-87e4-42793401bfd4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:47.446718 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd350665-cbb2-4abf-953b-cf97e421cf56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e58f65be55ec400e8fdf7bea456ac245)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:47.449187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd350665-cbb2-4abf-953b-cf97e421cf56 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:47.464918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa974d-54ff-48c4-a3c6-2f3d1b3d983d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:47.466157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa974d-54ff-48c4-a3c6-2f3d1b3d983d service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:47.466655 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa974d-54ff-48c4-a3c6-2f3d1b3d983d service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:47.467148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa974d-54ff-48c4-a3c6-2f3d1b3d983d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:47.485064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa974d-54ff-48c4-a3c6-2f3d1b3d983d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:47.489406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa974d-54ff-48c4-a3c6-2f3d1b3d983d service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:47.516118 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 901/1792] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:47.535693 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fd350665-cbb2-4abf-953b-cf97e421cf56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e58f65be55ec400e8fdf7bea456ac245', 'request_id': 'req-fd350665-cbb2-4abf-953b-cf97e421cf56'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:47.544112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fd350665-cbb2-4abf-953b-cf97e421cf56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e58f65be55ec400e8fdf7bea456ac245', 'request_id': 'req-fd350665-cbb2-4abf-953b-cf97e421cf56'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:47.553101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fd350665-cbb2-4abf-953b-cf97e421cf56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e58f65be55ec400e8fdf7bea456ac245', 'request_id': 'req-fd350665-cbb2-4abf-953b-cf97e421cf56'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:47.576601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fd350665-cbb2-4abf-953b-cf97e421cf56 admin admin] The token cache is being invalidate because project e58f65be55ec400e8fdf7bea456ac245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:47.577087 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fd350665-cbb2-4abf-953b-cf97e421cf56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:47.578525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fd350665-cbb2-4abf-953b-cf97e421cf56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:47.580449 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 892/1793] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:47 2022] DELETE /identity/v3/projects/e58f65be55ec400e8fdf7bea456ac245 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:47.585420 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66f2ff-a954-4c05-ac85-38b72211c1dd None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:47.586472 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66f2ff-a954-4c05-ac85-38b72211c1dd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:47.587309 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66f2ff-a954-4c05-ac85-38b72211c1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:47.615445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ac66f2ff-a954-4c05-ac85-38b72211c1dd None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:47.621060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ac66f2ff-a954-4c05-ac85-38b72211c1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:47.719569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 902/1794] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:47.728058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270d1e5-478a-4e59-92e4-2233e8417f76 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:47.728491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270d1e5-478a-4e59-92e4-2233e8417f76 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:47.729156 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270d1e5-478a-4e59-92e4-2233e8417f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:47.757546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1270d1e5-478a-4e59-92e4-2233e8417f76 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:47.759040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1270d1e5-478a-4e59-92e4-2233e8417f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:47.768691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 893/1795] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:47.774419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66f2ff-a954-4c05-ac85-38b72211c1dd None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:47.806300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66f2ff-a954-4c05-ac85-38b72211c1dd None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:47.815166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de7706-ad4e-4adb-b948-9ddbde7b31ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:47.815793 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de7706-ad4e-4adb-b948-9ddbde7b31ce admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:47.816070 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de7706-ad4e-4adb-b948-9ddbde7b31ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:47.816370 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de7706-ad4e-4adb-b948-9ddbde7b31ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:47.817273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de7706-ad4e-4adb-b948-9ddbde7b31ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:47.819899 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de7706-ad4e-4adb-b948-9ddbde7b31ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:47.819899 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de7706-ad4e-4adb-b948-9ddbde7b31ce admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:47.855236 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 903/1796] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:47.856181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270d1e5-478a-4e59-92e4-2233e8417f76 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:47.868202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de7706-ad4e-4adb-b948-9ddbde7b31ce admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:47.871324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-16de7706-ad4e-4adb-b948-9ddbde7b31ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:47.872649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270d1e5-478a-4e59-92e4-2233e8417f76 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:47.880062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b4b35-384e-4938-be88-e4cc29a3575e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:47.880798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b4b35-384e-4938-be88-e4cc29a3575e admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:47.881087 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b4b35-384e-4938-be88-e4cc29a3575e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:47.881483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b4b35-384e-4938-be88-e4cc29a3575e admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:47.882260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b4b35-384e-4938-be88-e4cc29a3575e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:47.884499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b4b35-384e-4938-be88-e4cc29a3575e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:47.888482 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de7706-ad4e-4adb-b948-9ddbde7b31ce admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:47.895373 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db369f1-e152-43e9-b4c8-54d0c2d80767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:47.896434 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db369f1-e152-43e9-b4c8-54d0c2d80767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:47.901820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db369f1-e152-43e9-b4c8-54d0c2d80767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:47.909167 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db369f1-e152-43e9-b4c8-54d0c2d80767 admin admin] PATH_INFO: `/v3/projects/3e471b8be3f94808851cb4c13608dfc0` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:47.911970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db369f1-e152-43e9-b4c8-54d0c2d80767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e471b8be3f94808851cb4c13608dfc0)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:47.917327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db369f1-e152-43e9-b4c8-54d0c2d80767 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:47.941099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 894/1797] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:47 2022] POST /identity/v3/projects => generated 369 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:47.942942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b1819-dd04-4be6-ba12-5ebe97c14592 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:47.943299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b1819-dd04-4be6-ba12-5ebe97c14592 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:47.943693 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b1819-dd04-4be6-ba12-5ebe97c14592 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:47.945153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 895/1798] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 7 07:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:55:47.946033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b1819-dd04-4be6-ba12-5ebe97c14592 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:47.958847 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:48.003124 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7db369f1-e152-43e9-b4c8-54d0c2d80767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e471b8be3f94808851cb4c13608dfc0', 'request_id': 'req-7db369f1-e152-43e9-b4c8-54d0c2d80767'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.014371 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7db369f1-e152-43e9-b4c8-54d0c2d80767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e471b8be3f94808851cb4c13608dfc0', 'request_id': 'req-7db369f1-e152-43e9-b4c8-54d0c2d80767'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.014987 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:48.025182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7db369f1-e152-43e9-b4c8-54d0c2d80767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e471b8be3f94808851cb4c13608dfc0', 'request_id': 'req-7db369f1-e152-43e9-b4c8-54d0c2d80767'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.035588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b1819-dd04-4be6-ba12-5ebe97c14592 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:48.045100 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7db369f1-e152-43e9-b4c8-54d0c2d80767 admin admin] The token cache is being invalidate because project 3e471b8be3f94808851cb4c13608dfc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:48.045407 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7db369f1-e152-43e9-b4c8-54d0c2d80767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.046181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7db369f1-e152-43e9-b4c8-54d0c2d80767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.051324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 904/1799] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:47 2022] DELETE /identity/v3/projects/3e471b8be3f94808851cb4c13608dfc0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:48.051324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f623a4-66ba-4f72-ae89-ebd4ecb92266 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.051324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f623a4-66ba-4f72-ae89-ebd4ecb92266 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.051324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f623a4-66ba-4f72-ae89-ebd4ecb92266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.074600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-32f623a4-66ba-4f72-ae89-ebd4ecb92266 None None] MFA Rules not processed for user `81d5689e9a344bafb74ca423ffd2b42a`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:48.076015 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-32f623a4-66ba-4f72-ae89-ebd4ecb92266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:48.107885 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c287b-f204-4a7d-850a-1452441cc30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:48.108815 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c287b-f204-4a7d-850a-1452441cc30a admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.109092 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c287b-f204-4a7d-850a-1452441cc30a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.109395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c287b-f204-4a7d-850a-1452441cc30a admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.110167 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c287b-f204-4a7d-850a-1452441cc30a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:48.112145 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c287b-f204-4a7d-850a-1452441cc30a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:48.140450 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 896/1800] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:47 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:48.141803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c287b-f204-4a7d-850a-1452441cc30a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:48.144397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d56c287b-f204-4a7d-850a-1452441cc30a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:48.161007 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 905/1801] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 7 07:55:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:48.161321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c287b-f204-4a7d-850a-1452441cc30a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:48.162350 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c23584-15ba-4749-9d9d-c9f54ae4fdbb None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.162577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c23584-15ba-4749-9d9d-c9f54ae4fdbb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.162857 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c23584-15ba-4749-9d9d-c9f54ae4fdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.167045 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:48.176123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.176123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.176123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2 admin admin] PATH_INFO: `/v3/projects/7796d628744b425893d3408c347f2b60` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.176123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7796d628744b425893d3408c347f2b60)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:48.176123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:48.195526 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-51c23584-15ba-4749-9d9d-c9f54ae4fdbb None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:48.196887 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-51c23584-15ba-4749-9d9d-c9f54ae4fdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:48.205957 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 906/1802] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:48.206909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c23584-15ba-4749-9d9d-c9f54ae4fdbb None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:48.213934 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-51c23584-15ba-4749-9d9d-c9f54ae4fdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:48.228063 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c23584-15ba-4749-9d9d-c9f54ae4fdbb None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:48.246729 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7796d628744b425893d3408c347f2b60', 'request_id': 'req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.254611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7796d628744b425893d3408c347f2b60', 'request_id': 'req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.262006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7796d628744b425893d3408c347f2b60', 'request_id': 'req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.277476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2 admin admin] The token cache is being invalidate because project 7796d628744b425893d3408c347f2b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:48.277844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.278512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.280203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 897/1803] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:48 2022] DELETE /identity/v3/projects/7796d628744b425893d3408c347f2b60 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:48.280962 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:48.284224 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:48.304491 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af131b4-b50d-470a-8e9b-60b88251fc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:48.305407 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af131b4-b50d-470a-8e9b-60b88251fc5a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.305693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af131b4-b50d-470a-8e9b-60b88251fc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.305968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af131b4-b50d-470a-8e9b-60b88251fc5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.306607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af131b4-b50d-470a-8e9b-60b88251fc5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:48.309992 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af131b4-b50d-470a-8e9b-60b88251fc5a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:48.321772 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 907/1804] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:48.323078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cec40-e540-44eb-96d8-25e044c6f633 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.323358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cec40-e540-44eb-96d8-25e044c6f633 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.323629 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cec40-e540-44eb-96d8-25e044c6f633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.349430 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e71cec40-e540-44eb-96d8-25e044c6f633 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:48.350695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e71cec40-e540-44eb-96d8-25e044c6f633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:48.361521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 908/1805] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:48.365423 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cec40-e540-44eb-96d8-25e044c6f633 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:48.366725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e71cec40-e540-44eb-96d8-25e044c6f633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:48.374743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:48.378764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cec40-e540-44eb-96d8-25e044c6f633 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:48.385472 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404649c1-bf92-4f44-b461-82e873fa4229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:48.385860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef993c5-6e3b-4c5f-9e2a-2776701beed2 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:48.386566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404649c1-bf92-4f44-b461-82e873fa4229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.387341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404649c1-bf92-4f44-b461-82e873fa4229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.387619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404649c1-bf92-4f44-b461-82e873fa4229 admin admin] PATH_INFO: `/v3/users/bc687840a22c428ba5df20a5d9703ec5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.388999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404649c1-bf92-4f44-b461-82e873fa4229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc687840a22c428ba5df20a5d9703ec5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:48.391081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404649c1-bf92-4f44-b461-82e873fa4229 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:48.392744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3b237-aab9-4241-953d-e0f63b1359ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:48.393695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3b237-aab9-4241-953d-e0f63b1359ee service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.396679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3b237-aab9-4241-953d-e0f63b1359ee service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.398055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3b237-aab9-4241-953d-e0f63b1359ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.401041 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-77a3b237-aab9-4241-953d-e0f63b1359ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:48.418124 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:48.418532 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3b237-aab9-4241-953d-e0f63b1359ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:48.422281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3b237-aab9-4241-953d-e0f63b1359ee service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:48.437209 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 898/1806] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:48.438138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3b237-aab9-4241-953d-e0f63b1359ee service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:48.439560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-77a3b237-aab9-4241-953d-e0f63b1359ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:48.454613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3b237-aab9-4241-953d-e0f63b1359ee service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:48.459503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-404649c1-bf92-4f44-b461-82e873fa4229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc687840a22c428ba5df20a5d9703ec5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.460333 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c57bd-f2ba-48c8-868c-3fea96b2c5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:48.461679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c57bd-f2ba-48c8-868c-3fea96b2c5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.462910 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c57bd-f2ba-48c8-868c-3fea96b2c5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.463273 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c57bd-f2ba-48c8-868c-3fea96b2c5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.463998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c57bd-f2ba-48c8-868c-3fea96b2c5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:48.466008 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c57bd-f2ba-48c8-868c-3fea96b2c5d8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:48.473741 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-404649c1-bf92-4f44-b461-82e873fa4229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc687840a22c428ba5df20a5d9703ec5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.479179 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 899/1807] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:48.480192 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c57bd-f2ba-48c8-868c-3fea96b2c5d8 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:48.488965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-404649c1-bf92-4f44-b461-82e873fa4229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc687840a22c428ba5df20a5d9703ec5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.489853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-404649c1-bf92-4f44-b461-82e873fa4229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc687840a22c428ba5df20a5d9703ec5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.494207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c57bd-f2ba-48c8-868c-3fea96b2c5d8 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:48.499480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-404649c1-bf92-4f44-b461-82e873fa4229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc687840a22c428ba5df20a5d9703ec5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.501564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094936a2-92b0-44c1-a24c-1981a1ff6b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:48.503403 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094936a2-92b0-44c1-a24c-1981a1ff6b0f admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.504531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094936a2-92b0-44c1-a24c-1981a1ff6b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.505110 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094936a2-92b0-44c1-a24c-1981a1ff6b0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.505934 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094936a2-92b0-44c1-a24c-1981a1ff6b0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:48.507096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094936a2-92b0-44c1-a24c-1981a1ff6b0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:48.508604 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094936a2-92b0-44c1-a24c-1981a1ff6b0f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:48.512033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 909/1808] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:48 2022] DELETE /identity/v3/users/bc687840a22c428ba5df20a5d9703ec5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:48.513018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404649c1-bf92-4f44-b461-82e873fa4229 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:48.514464 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-404649c1-bf92-4f44-b461-82e873fa4229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:48.545656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 900/1809] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:48.550203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094936a2-92b0-44c1-a24c-1981a1ff6b0f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:48.552504 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-094936a2-92b0-44c1-a24c-1981a1ff6b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:48.573858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094936a2-92b0-44c1-a24c-1981a1ff6b0f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:48.605900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404649c1-bf92-4f44-b461-82e873fa4229 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:48.613732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1ecf0-5488-4b2e-8210-a728d20d1c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:48.614431 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1ecf0-5488-4b2e-8210-a728d20d1c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.614676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1ecf0-5488-4b2e-8210-a728d20d1c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.615044 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1ecf0-5488-4b2e-8210-a728d20d1c0d admin admin] PATH_INFO: `/v3/projects/58f70c9b205b46debdb421351bf32ef6/users/7bcb6f5305214a2ba5764f02c58f498e/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.631581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1ecf0-5488-4b2e-8210-a728d20d1c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=58f70c9b205b46debdb421351bf32ef6, user_id=7bcb6f5305214a2ba5764f02c58f498e, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:48.635762 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1ecf0-5488-4b2e-8210-a728d20d1c0d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:48.651835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 910/1810] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:48 2022] PUT /identity/v3/projects/58f70c9b205b46debdb421351bf32ef6/users/7bcb6f5305214a2ba5764f02c58f498e/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:48.663390 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f16b4b-5b28-4bb5-bc79-102e05335c99 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.663676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f16b4b-5b28-4bb5-bc79-102e05335c99 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.664063 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f16b4b-5b28-4bb5-bc79-102e05335c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.680028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b6e76-1ee6-4179-b9e9-7042ba481243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:48.680763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b6e76-1ee6-4179-b9e9-7042ba481243 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.681105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b6e76-1ee6-4179-b9e9-7042ba481243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.681445 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b6e76-1ee6-4179-b9e9-7042ba481243 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.682137 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b6e76-1ee6-4179-b9e9-7042ba481243 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:48.684168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b6e76-1ee6-4179-b9e9-7042ba481243 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:48.710553 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-47f16b4b-5b28-4bb5-bc79-102e05335c99 None None] MFA Rules not processed for user `7bcb6f5305214a2ba5764f02c58f498e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:48.713182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-47f16b4b-5b28-4bb5-bc79-102e05335c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:48.724159 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 901/1811] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:48 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:48.725314 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b6e76-1ee6-4179-b9e9-7042ba481243 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:48.802083 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b6e76-1ee6-4179-b9e9-7042ba481243 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:48.804453 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 911/1812] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:48.809166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f16b4b-5b28-4bb5-bc79-102e05335c99 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:48.811254 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211cc1e-08dc-4a67-b20d-da7d9388c9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:48.811941 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211cc1e-08dc-4a67-b20d-da7d9388c9c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.812310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211cc1e-08dc-4a67-b20d-da7d9388c9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.812647 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211cc1e-08dc-4a67-b20d-da7d9388c9c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.813344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211cc1e-08dc-4a67-b20d-da7d9388c9c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:48.815366 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211cc1e-08dc-4a67-b20d-da7d9388c9c4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:48.824342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:48.848547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 902/1813] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:48 2022] POST /identity/v3/users => generated 290 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:48.855100 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211cc1e-08dc-4a67-b20d-da7d9388c9c4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:48.874539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211cc1e-08dc-4a67-b20d-da7d9388c9c4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:48.880456 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4f8b0-f7df-47ba-b7c3-d8080d4958b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:48.881222 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4f8b0-f7df-47ba-b7c3-d8080d4958b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.881572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4f8b0-f7df-47ba-b7c3-d8080d4958b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.881844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4f8b0-f7df-47ba-b7c3-d8080d4958b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.882930 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4f8b0-f7df-47ba-b7c3-d8080d4958b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:48.885829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4f8b0-f7df-47ba-b7c3-d8080d4958b0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:48.901226 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 903/1814] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:55:48.902159 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4f8b0-f7df-47ba-b7c3-d8080d4958b0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:48.903798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9ac4f8b0-f7df-47ba-b7c3-d8080d4958b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:48.917052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4f8b0-f7df-47ba-b7c3-d8080d4958b0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:48.918905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f16b4b-5b28-4bb5-bc79-102e05335c99 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:48.921799 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290022a5-e819-4993-87e8-e3b9036ffc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:48.922349 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290022a5-e819-4993-87e8-e3b9036ffc01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.922672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290022a5-e819-4993-87e8-e3b9036ffc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.922884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290022a5-e819-4993-87e8-e3b9036ffc01 admin admin] PATH_INFO: `/v3/projects/bb486dfc42a34848ba2fef9ee293ae0c` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.923968 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290022a5-e819-4993-87e8-e3b9036ffc01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb486dfc42a34848ba2fef9ee293ae0c)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:48.924387 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6cee57-2056-4484-b085-c0db71b7bed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:48.925146 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cee57-2056-4484-b085-c0db71b7bed4 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:48.925473 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cee57-2056-4484-b085-c0db71b7bed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:48.925815 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cee57-2056-4484-b085-c0db71b7bed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:48.926140 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290022a5-e819-4993-87e8-e3b9036ffc01 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:48.926570 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6cee57-2056-4484-b085-c0db71b7bed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:48.929140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6cee57-2056-4484-b085-c0db71b7bed4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:48.960740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 912/1815] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:48 2022] POST /identity/v3/projects => generated 369 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:48.963897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:48.965651 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6cee57-2056-4484-b085-c0db71b7bed4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:48.982808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-290022a5-e819-4993-87e8-e3b9036ffc01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb486dfc42a34848ba2fef9ee293ae0c', 'request_id': 'req-290022a5-e819-4993-87e8-e3b9036ffc01'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.988412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-290022a5-e819-4993-87e8-e3b9036ffc01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb486dfc42a34848ba2fef9ee293ae0c', 'request_id': 'req-290022a5-e819-4993-87e8-e3b9036ffc01'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:48.996583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-290022a5-e819-4993-87e8-e3b9036ffc01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb486dfc42a34848ba2fef9ee293ae0c', 'request_id': 'req-290022a5-e819-4993-87e8-e3b9036ffc01'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:49.012711 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-290022a5-e819-4993-87e8-e3b9036ffc01 admin admin] The token cache is being invalidate because project bb486dfc42a34848ba2fef9ee293ae0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:49.013113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-290022a5-e819-4993-87e8-e3b9036ffc01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:49.014039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-290022a5-e819-4993-87e8-e3b9036ffc01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:49.015981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 904/1816] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:48 2022] DELETE /identity/v3/projects/bb486dfc42a34848ba2fef9ee293ae0c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:49.017191 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290022a5-e819-4993-87e8-e3b9036ffc01 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:49.018521 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-290022a5-e819-4993-87e8-e3b9036ffc01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:49.038746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6cee57-2056-4484-b085-c0db71b7bed4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:49.085184 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290022a5-e819-4993-87e8-e3b9036ffc01 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:49.090271 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ee3d3-d358-464f-a222-15f1de04f7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:49.090881 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ee3d3-d358-464f-a222-15f1de04f7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:49.091163 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ee3d3-d358-464f-a222-15f1de04f7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:49.091437 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ee3d3-d358-464f-a222-15f1de04f7b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:49.092065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ee3d3-d358-464f-a222-15f1de04f7b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:49.094017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ee3d3-d358-464f-a222-15f1de04f7b7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:49.098241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7a92e-8432-4d21-acc1-e9819e28d10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:49.098930 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7a92e-8432-4d21-acc1-e9819e28d10c admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:49.099277 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7a92e-8432-4d21-acc1-e9819e28d10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:49.099617 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7a92e-8432-4d21-acc1-e9819e28d10c admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:49.100482 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f7a92e-8432-4d21-acc1-e9819e28d10c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:49.102697 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f7a92e-8432-4d21-acc1-e9819e28d10c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:49.113966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 913/1817] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:49.119082 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957d6e9-0125-4360-93dc-a6fa24cce12e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:49.119254 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 905/1818] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:49 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:49.119453 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957d6e9-0125-4360-93dc-a6fa24cce12e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:49.119607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957d6e9-0125-4360-93dc-a6fa24cce12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:49.120439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ee3d3-d358-464f-a222-15f1de04f7b7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:49.124095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c69ee3d3-d358-464f-a222-15f1de04f7b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:49.137686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ee3d3-d358-464f-a222-15f1de04f7b7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:49.140041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e957d6e9-0125-4360-93dc-a6fa24cce12e None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:49.141276 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e957d6e9-0125-4360-93dc-a6fa24cce12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:49.142288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76db238-9115-4831-bfaf-dfda6b5aa5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:49.142932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76db238-9115-4831-bfaf-dfda6b5aa5d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:49.143207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76db238-9115-4831-bfaf-dfda6b5aa5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:49.143474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76db238-9115-4831-bfaf-dfda6b5aa5d3 admin admin] PATH_INFO: `/v3/projects/26e31244e9d54da2b515c612c3b9c650/users/5971b7f2ed414369b04769bed43c5f50/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:49.149581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 914/1819] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:49.150420 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957d6e9-0125-4360-93dc-a6fa24cce12e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:49.160768 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957d6e9-0125-4360-93dc-a6fa24cce12e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:49.163017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76db238-9115-4831-bfaf-dfda6b5aa5d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26e31244e9d54da2b515c612c3b9c650, user_id=5971b7f2ed414369b04769bed43c5f50, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:49.165504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529252f-aab1-4a19-9284-85d47f02bf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:49.165926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76db238-9115-4831-bfaf-dfda6b5aa5d3 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:49.166213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529252f-aab1-4a19-9284-85d47f02bf8e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:49.166558 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529252f-aab1-4a19-9284-85d47f02bf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:49.166900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529252f-aab1-4a19-9284-85d47f02bf8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:49.167603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529252f-aab1-4a19-9284-85d47f02bf8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:49.169248 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529252f-aab1-4a19-9284-85d47f02bf8e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:49.180710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 906/1820] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:49 2022] PUT /identity/v3/projects/26e31244e9d54da2b515c612c3b9c650/users/5971b7f2ed414369b04769bed43c5f50/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:49.182434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae93fcf-3266-48cd-afe6-1e6cc873ce6c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:49.182855 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae93fcf-3266-48cd-afe6-1e6cc873ce6c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:49.183263 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae93fcf-3266-48cd-afe6-1e6cc873ce6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:49.188144 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 915/1821] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:49.191018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bff6e-6b63-493d-8f78-129af9249c01 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:49.191299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bff6e-6b63-493d-8f78-129af9249c01 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:49.191825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bff6e-6b63-493d-8f78-129af9249c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:49.208926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-aae93fcf-3266-48cd-afe6-1e6cc873ce6c None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:49.210130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-aae93fcf-3266-48cd-afe6-1e6cc873ce6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:49.219835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9a7bff6e-6b63-493d-8f78-129af9249c01 None None] MFA Rules not processed for user `5971b7f2ed414369b04769bed43c5f50`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:49.221069 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9a7bff6e-6b63-493d-8f78-129af9249c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:49.236249 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:49.277047 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 907/1822] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:49.277671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae93fcf-3266-48cd-afe6-1e6cc873ce6c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:49.287694 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae93fcf-3266-48cd-afe6-1e6cc873ce6c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:49.295256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540044e-7ddb-48b4-8d4e-6e8241a872d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:49.295919 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540044e-7ddb-48b4-8d4e-6e8241a872d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:49.296223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540044e-7ddb-48b4-8d4e-6e8241a872d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:49.296496 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540044e-7ddb-48b4-8d4e-6e8241a872d7 admin admin] PATH_INFO: `/v3/projects/91b3d41051a34a9a9a805fdfd0f469bf/users/5f42307bfccc4a86b35ae60a1c6f56c1/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:49.306579 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 916/1823] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:55:49.307526 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bff6e-6b63-493d-8f78-129af9249c01 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:49.318914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540044e-7ddb-48b4-8d4e-6e8241a872d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91b3d41051a34a9a9a805fdfd0f469bf, user_id=5f42307bfccc4a86b35ae60a1c6f56c1, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:49.320959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bff6e-6b63-493d-8f78-129af9249c01 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:49.322260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540044e-7ddb-48b4-8d4e-6e8241a872d7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:49.325621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=6,keystone:SELECT=88,keystone:INSERT=6 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:49.326391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56c843-56cf-41f1-a3fa-51142c1468bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:49.327076 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56c843-56cf-41f1-a3fa-51142c1468bf admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:49.327446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56c843-56cf-41f1-a3fa-51142c1468bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:49.327772 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56c843-56cf-41f1-a3fa-51142c1468bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:49.328605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc56c843-56cf-41f1-a3fa-51142c1468bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:49.328924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc56c843-56cf-41f1-a3fa-51142c1468bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:49.330770 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc56c843-56cf-41f1-a3fa-51142c1468bf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:49.341546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 908/1824] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:49 2022] PUT /identity/v3/projects/91b3d41051a34a9a9a805fdfd0f469bf/users/5f42307bfccc4a86b35ae60a1c6f56c1/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:49.342609 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540044e-7ddb-48b4-8d4e-6e8241a872d7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:49.348026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 917/1825] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:49.348991 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56c843-56cf-41f1-a3fa-51142c1468bf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:49.362314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56c843-56cf-41f1-a3fa-51142c1468bf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:49.413430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540044e-7ddb-48b4-8d4e-6e8241a872d7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:49.426097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc54609-d5a0-424b-bf45-46c6dd67b5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:49.427010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc54609-d5a0-424b-bf45-46c6dd67b5b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:49.427284 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc54609-d5a0-424b-bf45-46c6dd67b5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:49.427623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc54609-d5a0-424b-bf45-46c6dd67b5b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:49.428460 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc54609-d5a0-424b-bf45-46c6dd67b5b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:49.444260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc54609-d5a0-424b-bf45-46c6dd67b5b5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:49.445071 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9eec3-4ae1-4ff2-b7cf-c5bdd6956cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:49.445071 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9eec3-4ae1-4ff2-b7cf-c5bdd6956cac admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:49.445071 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9eec3-4ae1-4ff2-b7cf-c5bdd6956cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:49.445071 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9eec3-4ae1-4ff2-b7cf-c5bdd6956cac admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:49.445507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9eec3-4ae1-4ff2-b7cf-c5bdd6956cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:49.445507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9eec3-4ae1-4ff2-b7cf-c5bdd6956cac admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:49.460183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 918/1826] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:49.460183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9eec3-4ae1-4ff2-b7cf-c5bdd6956cac admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:49.465608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 909/1827] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:49 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:49.466622 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc54609-d5a0-424b-bf45-46c6dd67b5b5 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:49.473472 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9eec3-4ae1-4ff2-b7cf-c5bdd6956cac admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:49.536355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa70ea-c221-419a-980d-f27e9aa87742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:49.536968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa70ea-c221-419a-980d-f27e9aa87742 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:49.537254 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa70ea-c221-419a-980d-f27e9aa87742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:49.537514 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa70ea-c221-419a-980d-f27e9aa87742 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:49.538174 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa70ea-c221-419a-980d-f27e9aa87742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:49.540053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc54609-d5a0-424b-bf45-46c6dd67b5b5 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:49.540983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa70ea-c221-419a-980d-f27e9aa87742 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:49.544934 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91cb41-ab67-4723-b716-fe38bfeeddf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:49.545604 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91cb41-ab67-4723-b716-fe38bfeeddf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:49.545881 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91cb41-ab67-4723-b716-fe38bfeeddf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:49.546154 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91cb41-ab67-4723-b716-fe38bfeeddf8 admin admin] PATH_INFO: `/v3/projects/91b3d41051a34a9a9a805fdfd0f469bf/users/5f42307bfccc4a86b35ae60a1c6f56c1/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:49.548516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91cb41-ab67-4723-b716-fe38bfeeddf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91b3d41051a34a9a9a805fdfd0f469bf, user_id=5f42307bfccc4a86b35ae60a1c6f56c1, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:49.551186 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91cb41-ab67-4723-b716-fe38bfeeddf8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:49.565199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 910/1828] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:49 2022] PUT /identity/v3/projects/91b3d41051a34a9a9a805fdfd0f469bf/users/5f42307bfccc4a86b35ae60a1c6f56c1/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:49.566040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91cb41-ab67-4723-b716-fe38bfeeddf8 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:49.585054 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 919/1829] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:49 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:49.587309 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97d00f-7931-46fb-83a1-e5449ff66166 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:49.587568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97d00f-7931-46fb-83a1-e5449ff66166 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:49.595517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97d00f-7931-46fb-83a1-e5449ff66166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:49.642477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-da97d00f-7931-46fb-83a1-e5449ff66166 None None] MFA Rules not processed for user `5f42307bfccc4a86b35ae60a1c6f56c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:49.644603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-da97d00f-7931-46fb-83a1-e5449ff66166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:49.651759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:49.692253 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91cb41-ab67-4723-b716-fe38bfeeddf8 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:49.736265 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:49.790316 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 920/1830] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:49.797382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97d00f-7931-46fb-83a1-e5449ff66166 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:49.808365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac2d03-c4df-4462-acea-d2825ef1dc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:49.809033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac2d03-c4df-4462-acea-d2825ef1dc1f admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:49.809382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac2d03-c4df-4462-acea-d2825ef1dc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:49.809658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac2d03-c4df-4462-acea-d2825ef1dc1f admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:49.810300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac2d03-c4df-4462-acea-d2825ef1dc1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:49.812280 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac2d03-c4df-4462-acea-d2825ef1dc1f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:49.857099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 911/1831] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:49 2022] POST /identity/v3/users => generated 296 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:49.858483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac2d03-c4df-4462-acea-d2825ef1dc1f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:49.860673 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-81ac2d03-c4df-4462-acea-d2825ef1dc1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:49.976331 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac2d03-c4df-4462-acea-d2825ef1dc1f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:49.977563 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97d00f-7931-46fb-83a1-e5449ff66166 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:49.981959 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871f508-454a-4d64-902f-00f1e97f2739 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:49.982759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871f508-454a-4d64-902f-00f1e97f2739 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:49.983090 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871f508-454a-4d64-902f-00f1e97f2739 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:49.983367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871f508-454a-4d64-902f-00f1e97f2739 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:50.011003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688eb37-2806-4a62-a13a-48377e00c4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:50.014067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688eb37-2806-4a62-a13a-48377e00c4cb admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:50.015266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688eb37-2806-4a62-a13a-48377e00c4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:50.015584 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688eb37-2806-4a62-a13a-48377e00c4cb admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:50.016340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e688eb37-2806-4a62-a13a-48377e00c4cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:50.023005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e688eb37-2806-4a62-a13a-48377e00c4cb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:50.056704 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 921/1832] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:49 2022] POST /identity/v3/users => generated 294 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:50.060722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688eb37-2806-4a62-a13a-48377e00c4cb admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:50.060877 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871f508-454a-4d64-902f-00f1e97f2739 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:50.062836 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871f508-454a-4d64-902f-00f1e97f2739 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:50.081271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688eb37-2806-4a62-a13a-48377e00c4cb admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:50.097998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 912/1833] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:50.097998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871f508-454a-4d64-902f-00f1e97f2739 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:50.099804 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5968e8-077a-4152-b069-07b386ad913f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:50.105718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5968e8-077a-4152-b069-07b386ad913f admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:50.106435 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5968e8-077a-4152-b069-07b386ad913f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:50.106973 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5968e8-077a-4152-b069-07b386ad913f admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:50.107838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5968e8-077a-4152-b069-07b386ad913f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:50.109975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5968e8-077a-4152-b069-07b386ad913f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:50.120879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871f508-454a-4d64-902f-00f1e97f2739 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:50.127499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056d517-3658-48ad-9c35-6c4128444f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:50.128294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056d517-3658-48ad-9c35-6c4128444f13 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:50.128405 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056d517-3658-48ad-9c35-6c4128444f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:50.128666 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056d517-3658-48ad-9c35-6c4128444f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:50.128865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 922/1834] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:50.129259 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056d517-3658-48ad-9c35-6c4128444f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:50.130597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5968e8-077a-4152-b069-07b386ad913f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:50.131509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056d517-3658-48ad-9c35-6c4128444f13 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:50.149224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5968e8-077a-4152-b069-07b386ad913f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:50.152668 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 913/1835] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:50.156216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056d517-3658-48ad-9c35-6c4128444f13 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:50.158865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e0166-9a64-45e7-b278-c7d8ec643976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:50.160529 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e0166-9a64-45e7-b278-c7d8ec643976 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:50.161213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e0166-9a64-45e7-b278-c7d8ec643976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:50.161699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e0166-9a64-45e7-b278-c7d8ec643976 admin admin] PATH_INFO: `/v3/projects/9c3f5e5d2bde492cb89fd66b5901b074/users/d33c436faebe4d0e9012ca89d42a895b/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:50.169178 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056d517-3658-48ad-9c35-6c4128444f13 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:50.174693 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9139a7-950e-4ca5-b10f-4a5d0e804890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:50.175394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9139a7-950e-4ca5-b10f-4a5d0e804890 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:50.175747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9139a7-950e-4ca5-b10f-4a5d0e804890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:50.176135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9139a7-950e-4ca5-b10f-4a5d0e804890 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:50.176843 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9139a7-950e-4ca5-b10f-4a5d0e804890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:50.178558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9139a7-950e-4ca5-b10f-4a5d0e804890 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:50.184224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e0166-9a64-45e7-b278-c7d8ec643976 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c3f5e5d2bde492cb89fd66b5901b074, user_id=d33c436faebe4d0e9012ca89d42a895b, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:50.188101 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e0166-9a64-45e7-b278-c7d8ec643976 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:50.192489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 914/1836] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:50.193365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9139a7-950e-4ca5-b10f-4a5d0e804890 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:50.196929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dc9139a7-950e-4ca5-b10f-4a5d0e804890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:50.211383 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9139a7-950e-4ca5-b10f-4a5d0e804890 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:50.224479 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 923/1837] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:50 2022] PUT /identity/v3/projects/9c3f5e5d2bde492cb89fd66b5901b074/users/d33c436faebe4d0e9012ca89d42a895b/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:50.225904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e0166-9a64-45e7-b278-c7d8ec643976 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:50.286744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05537ca-1047-448f-b336-96a643d8ea0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:50.287494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05537ca-1047-448f-b336-96a643d8ea0c service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:50.287776 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05537ca-1047-448f-b336-96a643d8ea0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:50.288050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05537ca-1047-448f-b336-96a643d8ea0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:50.290527 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d05537ca-1047-448f-b336-96a643d8ea0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:50.319721 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:50.326750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e0166-9a64-45e7-b278-c7d8ec643976 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:50.351661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1d315-e617-4f3c-b57e-31bae35b6c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:50.357937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1d315-e617-4f3c-b57e-31bae35b6c46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:50.357937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1d315-e617-4f3c-b57e-31bae35b6c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:50.357937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1d315-e617-4f3c-b57e-31bae35b6c46 admin admin] PATH_INFO: `/v3/projects/9272743ba2604b9faf1a62ea671ff3e4/users/d095f1b098934128b59be9e79b7e701c/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:50.399386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:50.400707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1d315-e617-4f3c-b57e-31bae35b6c46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9272743ba2604b9faf1a62ea671ff3e4, user_id=d095f1b098934128b59be9e79b7e701c, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:50.405707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1d315-e617-4f3c-b57e-31bae35b6c46 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:50.428603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05537ca-1047-448f-b336-96a643d8ea0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:50.431527 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05537ca-1047-448f-b336-96a643d8ea0c service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:50.452476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 915/1838] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:50.454057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05537ca-1047-448f-b336-96a643d8ea0c service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:50.456223 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 924/1839] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:50 2022] PUT /identity/v3/projects/9272743ba2604b9faf1a62ea671ff3e4/users/d095f1b098934128b59be9e79b7e701c/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:50.466726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05537ca-1047-448f-b336-96a643d8ea0c service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:50.469036 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f721c5-23a8-4b46-826e-bf66eba9fd0d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:50.469320 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f721c5-23a8-4b46-826e-bf66eba9fd0d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:50.469657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f721c5-23a8-4b46-826e-bf66eba9fd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:50.528979 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b2f721c5-23a8-4b46-826e-bf66eba9fd0d None None] MFA Rules not processed for user `d095f1b098934128b59be9e79b7e701c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:50.530676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b2f721c5-23a8-4b46-826e-bf66eba9fd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:50.589005 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b717157-4bb6-4b47-8952-6e351a25d786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:50.589699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b717157-4bb6-4b47-8952-6e351a25d786 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:50.589981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b717157-4bb6-4b47-8952-6e351a25d786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:50.590257 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b717157-4bb6-4b47-8952-6e351a25d786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:50.591215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b717157-4bb6-4b47-8952-6e351a25d786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:50.592868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b717157-4bb6-4b47-8952-6e351a25d786 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:50.606626 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 916/1840] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:50.610660 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b717157-4bb6-4b47-8952-6e351a25d786 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:50.627120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b717157-4bb6-4b47-8952-6e351a25d786 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:50.640416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61633e8-011f-47ef-80bf-1f5338f71ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:50.641069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61633e8-011f-47ef-80bf-1f5338f71ded admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:50.641345 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61633e8-011f-47ef-80bf-1f5338f71ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:50.641890 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61633e8-011f-47ef-80bf-1f5338f71ded admin admin] PATH_INFO: `/v3/projects/9c3f5e5d2bde492cb89fd66b5901b074/users/d33c436faebe4d0e9012ca89d42a895b/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:50.644553 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61633e8-011f-47ef-80bf-1f5338f71ded admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c3f5e5d2bde492cb89fd66b5901b074, user_id=d33c436faebe4d0e9012ca89d42a895b, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:50.651032 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61633e8-011f-47ef-80bf-1f5338f71ded admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:50.674060 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 917/1841] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:50 2022] PUT /identity/v3/projects/9c3f5e5d2bde492cb89fd66b5901b074/users/d33c436faebe4d0e9012ca89d42a895b/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:50.676524 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d484be1-97af-4c71-aad5-58ecb06d5542 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:50.677125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d484be1-97af-4c71-aad5-58ecb06d5542 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:50.677550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d484be1-97af-4c71-aad5-58ecb06d5542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:50.694759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 925/1842] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:50.698346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f721c5-23a8-4b46-826e-bf66eba9fd0d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:50.719968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f721c5-23a8-4b46-826e-bf66eba9fd0d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:50.727800 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-5d484be1-97af-4c71-aad5-58ecb06d5542 None None] MFA Rules not processed for user `d33c436faebe4d0e9012ca89d42a895b`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:50.729317 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5d484be1-97af-4c71-aad5-58ecb06d5542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:50.869916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 918/1843] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:50.879372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d484be1-97af-4c71-aad5-58ecb06d5542 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:50.895167 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d60b1-8358-4035-ae44-209a7351fc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:50.896195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d60b1-8358-4035-ae44-209a7351fc95 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:50.897404 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d60b1-8358-4035-ae44-209a7351fc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:50.897801 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d60b1-8358-4035-ae44-209a7351fc95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:50.899228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d60b1-8358-4035-ae44-209a7351fc95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:50.901892 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d60b1-8358-4035-ae44-209a7351fc95 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:50.948410 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 926/1844] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:50 2022] POST /identity/v3/projects => generated 387 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:50.949638 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d60b1-8358-4035-ae44-209a7351fc95 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:50.969285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d484be1-97af-4c71-aad5-58ecb06d5542 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:50.980719 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:51.098876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf50661-126b-468f-ae5d-737ca1a6281e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:51.101178 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf50661-126b-468f-ae5d-737ca1a6281e service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:51.102824 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf50661-126b-468f-ae5d-737ca1a6281e service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:51.103924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf50661-126b-468f-ae5d-737ca1a6281e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:51.110319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:51.121983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d60b1-8358-4035-ae44-209a7351fc95 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:51.129454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff561f-4566-494d-93eb-f8e02e264293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:51.130313 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff561f-4566-494d-93eb-f8e02e264293 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:51.130768 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff561f-4566-494d-93eb-f8e02e264293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:51.131120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff561f-4566-494d-93eb-f8e02e264293 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:51.131992 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff561f-4566-494d-93eb-f8e02e264293 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:51.135206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff561f-4566-494d-93eb-f8e02e264293 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:51.172786 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 927/1845] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:50 2022] POST /identity/v3/users => generated 300 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:51.174105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff561f-4566-494d-93eb-f8e02e264293 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:51.186069 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff561f-4566-494d-93eb-f8e02e264293 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:51.192431 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dfb1d-ea53-4007-a6a0-0258e8118de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:51.193140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dfb1d-ea53-4007-a6a0-0258e8118de0 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:51.193423 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dfb1d-ea53-4007-a6a0-0258e8118de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:51.193782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dfb1d-ea53-4007-a6a0-0258e8118de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:51.194439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7dfb1d-ea53-4007-a6a0-0258e8118de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:51.196109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7dfb1d-ea53-4007-a6a0-0258e8118de0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:51.204008 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf50661-126b-468f-ae5d-737ca1a6281e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:51.206168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf50661-126b-468f-ae5d-737ca1a6281e service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:51.210592 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 928/1846] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:51.215236 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dfb1d-ea53-4007-a6a0-0258e8118de0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:51.224251 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 919/1847] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:51.233588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dfb1d-ea53-4007-a6a0-0258e8118de0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:51.238598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120197c6-db36-40c3-bb55-2a75b8978d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:51.239265 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120197c6-db36-40c3-bb55-2a75b8978d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:51.239541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120197c6-db36-40c3-bb55-2a75b8978d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:51.239814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120197c6-db36-40c3-bb55-2a75b8978d56 admin admin] PATH_INFO: `/v3/projects/c1af57535ad042c490c73c2aeeced41c/users/aa05b328c8794e8295ef71dadd33feb2/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:51.258419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120197c6-db36-40c3-bb55-2a75b8978d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1af57535ad042c490c73c2aeeced41c, user_id=aa05b328c8794e8295ef71dadd33feb2, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:51.261367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120197c6-db36-40c3-bb55-2a75b8978d56 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:51.281024 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 929/1848] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:51 2022] PUT /identity/v3/projects/c1af57535ad042c490c73c2aeeced41c/users/aa05b328c8794e8295ef71dadd33feb2/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:51.284777 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf50661-126b-468f-ae5d-737ca1a6281e service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:51.296527 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf50661-126b-468f-ae5d-737ca1a6281e service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:51.365637 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efce58-2b87-4e0a-b55f-533b43e6f199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:51.366249 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efce58-2b87-4e0a-b55f-533b43e6f199 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:51.366528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efce58-2b87-4e0a-b55f-533b43e6f199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:51.366805 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efce58-2b87-4e0a-b55f-533b43e6f199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:51.367760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2efce58-2b87-4e0a-b55f-533b43e6f199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:51.369594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2efce58-2b87-4e0a-b55f-533b43e6f199 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:51.379814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 920/1849] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:51.384516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120197c6-db36-40c3-bb55-2a75b8978d56 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:51.395497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120197c6-db36-40c3-bb55-2a75b8978d56 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:51.401099 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869d52c-c656-48c9-853c-74f5e2ea8e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:51.401811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869d52c-c656-48c9-853c-74f5e2ea8e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:51.402089 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869d52c-c656-48c9-853c-74f5e2ea8e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:51.402361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869d52c-c656-48c9-853c-74f5e2ea8e4d admin admin] PATH_INFO: `/v3/projects/c1af57535ad042c490c73c2aeeced41c/users/aa05b328c8794e8295ef71dadd33feb2/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:51.404302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869d52c-c656-48c9-853c-74f5e2ea8e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1af57535ad042c490c73c2aeeced41c, user_id=aa05b328c8794e8295ef71dadd33feb2, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:51.406938 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869d52c-c656-48c9-853c-74f5e2ea8e4d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:51.423755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 930/1850] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:51 2022] PUT /identity/v3/projects/c1af57535ad042c490c73c2aeeced41c/users/aa05b328c8794e8295ef71dadd33feb2/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:51.434559 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5444e-f652-4381-8875-9733b095afd8 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:51.434841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5444e-f652-4381-8875-9733b095afd8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:51.435144 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5444e-f652-4381-8875-9733b095afd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:51.465305 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d0f5444e-f652-4381-8875-9733b095afd8 None None] MFA Rules not processed for user `aa05b328c8794e8295ef71dadd33feb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:51.466516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d0f5444e-f652-4381-8875-9733b095afd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:51.516029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:51.545888 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 921/1851] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:51.556958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869d52c-c656-48c9-853c-74f5e2ea8e4d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:51.574572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5444e-f652-4381-8875-9733b095afd8 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:51.612724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869d52c-c656-48c9-853c-74f5e2ea8e4d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:51.616823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db4b07-a023-4fa4-901d-21c4e5b3a57f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:51.617560 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db4b07-a023-4fa4-901d-21c4e5b3a57f service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:51.617842 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db4b07-a023-4fa4-901d-21c4e5b3a57f service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:51.618113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db4b07-a023-4fa4-901d-21c4e5b3a57f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:51.627198 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db4b07-a023-4fa4-901d-21c4e5b3a57f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:51.628867 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db4b07-a023-4fa4-901d-21c4e5b3a57f service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:51.636793 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5444e-f652-4381-8875-9733b095afd8 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:51.639227 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 931/1852] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:51 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:51.641439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8d0cf-baa4-4185-b241-df0b37dfde6b tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33c436faebe4d0e9012ca89d42a895b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c3f5e5d2bde492cb89fd66b5901b074', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:51.642047 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8d0cf-baa4-4185-b241-df0b37dfde6b tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:51.642337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8d0cf-baa4-4185-b241-df0b37dfde6b tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:51.642613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8d0cf-baa4-4185-b241-df0b37dfde6b tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:51.643256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8d0cf-baa4-4185-b241-df0b37dfde6b tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:51.643562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8d0cf-baa4-4185-b241-df0b37dfde6b tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:51.645210 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8d0cf-baa4-4185-b241-df0b37dfde6b tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:51.660793 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 922/1853] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:55:51.662490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db4b07-a023-4fa4-901d-21c4e5b3a57f service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:51.673111 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db4b07-a023-4fa4-901d-21c4e5b3a57f service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:51.679256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3b54a-a8bd-4d78-80fb-5d50f45f2b37 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33c436faebe4d0e9012ca89d42a895b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c3f5e5d2bde492cb89fd66b5901b074', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:51.679975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3b54a-a8bd-4d78-80fb-5d50f45f2b37 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:51.680321 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3b54a-a8bd-4d78-80fb-5d50f45f2b37 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:51.680597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3b54a-a8bd-4d78-80fb-5d50f45f2b37 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:51.681364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3b54a-a8bd-4d78-80fb-5d50f45f2b37 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:51.683312 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3b54a-a8bd-4d78-80fb-5d50f45f2b37 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:51.714928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 932/1854] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:51 2022] POST /identity/v3/projects => generated 354 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:51.716171 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8d0cf-baa4-4185-b241-df0b37dfde6b tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:51.726956 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8d0cf-baa4-4185-b241-df0b37dfde6b tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:51.781234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3b54a-a8bd-4d78-80fb-5d50f45f2b37 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:51.787179 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b0b24-3285-431e-b87d-c1aaa7935ab2 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33c436faebe4d0e9012ca89d42a895b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c3f5e5d2bde492cb89fd66b5901b074', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:51.787790 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b0b24-3285-431e-b87d-c1aaa7935ab2 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:51.788069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b0b24-3285-431e-b87d-c1aaa7935ab2 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:51.788369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b0b24-3285-431e-b87d-c1aaa7935ab2 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:51.789684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b0b24-3285-431e-b87d-c1aaa7935ab2 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:51.790238 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b0b24-3285-431e-b87d-c1aaa7935ab2 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:51.791853 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b0b24-3285-431e-b87d-c1aaa7935ab2 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:51.803250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 923/1855] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:51.851278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b0b24-3285-431e-b87d-c1aaa7935ab2 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:51.854516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3b54a-a8bd-4d78-80fb-5d50f45f2b37 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:51.859785 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b421fd9-d808-467c-a639-8e77f11c5333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:51.860458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b421fd9-d808-467c-a639-8e77f11c5333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:51.860735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b421fd9-d808-467c-a639-8e77f11c5333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:51.861015 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b421fd9-d808-467c-a639-8e77f11c5333 admin admin] PATH_INFO: `/v3/users/d095f1b098934128b59be9e79b7e701c` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:51.862283 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b421fd9-d808-467c-a639-8e77f11c5333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d095f1b098934128b59be9e79b7e701c)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:51.864220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b421fd9-d808-467c-a639-8e77f11c5333 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:51.868803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b0b24-3285-431e-b87d-c1aaa7935ab2 tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:51.873852 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31809a16-46af-4fbe-8694-36cef81cf4bc tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd33c436faebe4d0e9012ca89d42a895b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c3f5e5d2bde492cb89fd66b5901b074', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:51.874552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31809a16-46af-4fbe-8694-36cef81cf4bc tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:51.874900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31809a16-46af-4fbe-8694-36cef81cf4bc tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:51.875241 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31809a16-46af-4fbe-8694-36cef81cf4bc tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] PATH_INFO: `/v3/projects/654944fe8a21462f9d258cfd577b1855` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:51.876493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31809a16-46af-4fbe-8694-36cef81cf4bc tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] RBAC: Authorizing `identity:delete_project(project_id=654944fe8a21462f9d258cfd577b1855)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:51.878459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31809a16-46af-4fbe-8694-36cef81cf4bc tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:51.890357 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:51.938476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2b421fd9-d808-467c-a639-8e77f11c5333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd095f1b098934128b59be9e79b7e701c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:51.945535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-31809a16-46af-4fbe-8694-36cef81cf4bc tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '654944fe8a21462f9d258cfd577b1855', 'request_id': 'req-31809a16-46af-4fbe-8694-36cef81cf4bc'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:51.952055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-31809a16-46af-4fbe-8694-36cef81cf4bc tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '654944fe8a21462f9d258cfd577b1855', 'request_id': 'req-31809a16-46af-4fbe-8694-36cef81cf4bc'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:51.956410 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2b421fd9-d808-467c-a639-8e77f11c5333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd095f1b098934128b59be9e79b7e701c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:51.959454 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-31809a16-46af-4fbe-8694-36cef81cf4bc tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '654944fe8a21462f9d258cfd577b1855', 'request_id': 'req-31809a16-46af-4fbe-8694-36cef81cf4bc'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:51.965829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:51.969747 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2b421fd9-d808-467c-a639-8e77f11c5333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd095f1b098934128b59be9e79b7e701c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:51.973058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2b421fd9-d808-467c-a639-8e77f11c5333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd095f1b098934128b59be9e79b7e701c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:51.982523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2b421fd9-d808-467c-a639-8e77f11c5333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd095f1b098934128b59be9e79b7e701c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:51.984288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-31809a16-46af-4fbe-8694-36cef81cf4bc tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] The token cache is being invalidate because project 654944fe8a21462f9d258cfd577b1855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:51.985046 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-31809a16-46af-4fbe-8694-36cef81cf4bc tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:51.985863 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-31809a16-46af-4fbe-8694-36cef81cf4bc tempest-VolumeQuotaClassesTest-138436696 tempest-VolumeQuotaClassesTest-138436696-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:51.987631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 924/1856] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:51 2022] DELETE /identity/v3/projects/654944fe8a21462f9d258cfd577b1855 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:51.990091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 933/1857] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:51 2022] DELETE /identity/v3/users/d095f1b098934128b59be9e79b7e701c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:51.997891 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b697dc-bb57-4263-a160-dd5e87bfd178 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:51.998444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b697dc-bb57-4263-a160-dd5e87bfd178 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:51.999183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b697dc-bb57-4263-a160-dd5e87bfd178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:52.030716 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-40b697dc-bb57-4263-a160-dd5e87bfd178 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:52.032299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-40b697dc-bb57-4263-a160-dd5e87bfd178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:52.122274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 925/1858] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:52.132811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b421fd9-d808-467c-a639-8e77f11c5333 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:52.134393 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2b421fd9-d808-467c-a639-8e77f11c5333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:52.206012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b421fd9-d808-467c-a639-8e77f11c5333 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:52.209931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04713d-41b0-47fc-9001-6ed64498979e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:52.212130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04713d-41b0-47fc-9001-6ed64498979e service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:52.212130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04713d-41b0-47fc-9001-6ed64498979e service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:52.212130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04713d-41b0-47fc-9001-6ed64498979e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:52.221311 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04713d-41b0-47fc-9001-6ed64498979e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:52.222737 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04713d-41b0-47fc-9001-6ed64498979e service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:52.235825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 934/1859] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 07:55:52.265312 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b697dc-bb57-4263-a160-dd5e87bfd178 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:52.267207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-40b697dc-bb57-4263-a160-dd5e87bfd178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:52.282955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b697dc-bb57-4263-a160-dd5e87bfd178 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:52.292794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52daf6ce-a0d5-4aa6-abb0-9538ab1f56f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:52.293745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52daf6ce-a0d5-4aa6-abb0-9538ab1f56f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:52.294258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52daf6ce-a0d5-4aa6-abb0-9538ab1f56f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:52.294708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52daf6ce-a0d5-4aa6-abb0-9538ab1f56f3 admin admin] PATH_INFO: `/v3/users/5971b7f2ed414369b04769bed43c5f50` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:52.296100 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52daf6ce-a0d5-4aa6-abb0-9538ab1f56f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5971b7f2ed414369b04769bed43c5f50)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:52.298532 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52daf6ce-a0d5-4aa6-abb0-9538ab1f56f3 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:52.405824 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-52daf6ce-a0d5-4aa6-abb0-9538ab1f56f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5971b7f2ed414369b04769bed43c5f50'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:52.406582 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-52daf6ce-a0d5-4aa6-abb0-9538ab1f56f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5971b7f2ed414369b04769bed43c5f50'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:52.426531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-52daf6ce-a0d5-4aa6-abb0-9538ab1f56f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5971b7f2ed414369b04769bed43c5f50'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:52.427776 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-52daf6ce-a0d5-4aa6-abb0-9538ab1f56f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5971b7f2ed414369b04769bed43c5f50'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:52.440371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-52daf6ce-a0d5-4aa6-abb0-9538ab1f56f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5971b7f2ed414369b04769bed43c5f50'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:52.467419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 926/1860] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:52 2022] DELETE /identity/v3/users/5971b7f2ed414369b04769bed43c5f50 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:52.471954 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19871f-a37c-4558-829b-7ad1f578c19a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:52.472279 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19871f-a37c-4558-829b-7ad1f578c19a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:52.472566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19871f-a37c-4558-829b-7ad1f578c19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:52.499490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5b19871f-a37c-4558-829b-7ad1f578c19a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:52.501288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5b19871f-a37c-4558-829b-7ad1f578c19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:52.622934 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 935/1861] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:52.626253 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52daf6ce-a0d5-4aa6-abb0-9538ab1f56f3 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:52.627981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-52daf6ce-a0d5-4aa6-abb0-9538ab1f56f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:52.639565 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19871f-a37c-4558-829b-7ad1f578c19a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:52.662017 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5b19871f-a37c-4558-829b-7ad1f578c19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:52.670644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52daf6ce-a0d5-4aa6-abb0-9538ab1f56f3 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:52.680589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafbacd-c04a-4ede-831c-075e76a0879f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:52.681945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafbacd-c04a-4ede-831c-075e76a0879f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:52.682597 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafbacd-c04a-4ede-831c-075e76a0879f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:52.682973 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafbacd-c04a-4ede-831c-075e76a0879f admin admin] PATH_INFO: `/v3/projects/9272743ba2604b9faf1a62ea671ff3e4` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:52.684563 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cafbacd-c04a-4ede-831c-075e76a0879f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9272743ba2604b9faf1a62ea671ff3e4)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:52.687332 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cafbacd-c04a-4ede-831c-075e76a0879f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:52.717299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:52.733348 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19871f-a37c-4558-829b-7ad1f578c19a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:52.738073 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7cafbacd-c04a-4ede-831c-075e76a0879f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9272743ba2604b9faf1a62ea671ff3e4', 'request_id': 'req-7cafbacd-c04a-4ede-831c-075e76a0879f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:52.745135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7cafbacd-c04a-4ede-831c-075e76a0879f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9272743ba2604b9faf1a62ea671ff3e4', 'request_id': 'req-7cafbacd-c04a-4ede-831c-075e76a0879f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:52.747449 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=17,keystone:SELECT=82,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:52.755647 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7cafbacd-c04a-4ede-831c-075e76a0879f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9272743ba2604b9faf1a62ea671ff3e4', 'request_id': 'req-7cafbacd-c04a-4ede-831c-075e76a0879f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:52.772736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7cafbacd-c04a-4ede-831c-075e76a0879f admin admin] The token cache is being invalidate because project 9272743ba2604b9faf1a62ea671ff3e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:52.773024 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7cafbacd-c04a-4ede-831c-075e76a0879f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:52.773705 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7cafbacd-c04a-4ede-831c-075e76a0879f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:52.776112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 927/1862] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:52 2022] DELETE /identity/v3/projects/9272743ba2604b9faf1a62ea671ff3e4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:52.779308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafbacd-c04a-4ede-831c-075e76a0879f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:52.780700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7cafbacd-c04a-4ede-831c-075e76a0879f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:52.801590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ca88f-f5a7-459d-8150-1f8b87cb8848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:52.802628 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ca88f-f5a7-459d-8150-1f8b87cb8848 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:52.803115 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ca88f-f5a7-459d-8150-1f8b87cb8848 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:52.803608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ca88f-f5a7-459d-8150-1f8b87cb8848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:52.806614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fa7ca88f-f5a7-459d-8150-1f8b87cb8848 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:52.891823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafbacd-c04a-4ede-831c-075e76a0879f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:52.897538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94cefa-e487-4e55-bbe9-2208e31e55d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:52.898994 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94cefa-e487-4e55-bbe9-2208e31e55d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:52.899277 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94cefa-e487-4e55-bbe9-2208e31e55d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:52.899655 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94cefa-e487-4e55-bbe9-2208e31e55d5 admin admin] PATH_INFO: `/v3/users/aa05b328c8794e8295ef71dadd33feb2` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:52.900873 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae94cefa-e487-4e55-bbe9-2208e31e55d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa05b328c8794e8295ef71dadd33feb2)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:52.902895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae94cefa-e487-4e55-bbe9-2208e31e55d5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:52.907684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ca88f-f5a7-459d-8150-1f8b87cb8848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:52.909275 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ca88f-f5a7-459d-8150-1f8b87cb8848 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:52.928629 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 936/1863] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:52.982154 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ae94cefa-e487-4e55-bbe9-2208e31e55d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa05b328c8794e8295ef71dadd33feb2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:52.984317 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ae94cefa-e487-4e55-bbe9-2208e31e55d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa05b328c8794e8295ef71dadd33feb2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.021892 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ae94cefa-e487-4e55-bbe9-2208e31e55d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa05b328c8794e8295ef71dadd33feb2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.023122 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ae94cefa-e487-4e55-bbe9-2208e31e55d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa05b328c8794e8295ef71dadd33feb2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.036670 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ae94cefa-e487-4e55-bbe9-2208e31e55d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa05b328c8794e8295ef71dadd33feb2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.073494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 928/1864] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:52 2022] DELETE /identity/v3/users/aa05b328c8794e8295ef71dadd33feb2 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:53.522916 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ca88f-f5a7-459d-8150-1f8b87cb8848 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:53.526725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fa7ca88f-f5a7-459d-8150-1f8b87cb8848 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:53.575622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ca88f-f5a7-459d-8150-1f8b87cb8848 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:53.673128 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf14f2-9fab-4032-8bcd-f72903ddea05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:53.674931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf14f2-9fab-4032-8bcd-f72903ddea05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:53.674931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf14f2-9fab-4032-8bcd-f72903ddea05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:53.674931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf14f2-9fab-4032-8bcd-f72903ddea05 admin admin] PATH_INFO: `/v3/projects/26e31244e9d54da2b515c612c3b9c650` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:53.675917 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf14f2-9fab-4032-8bcd-f72903ddea05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26e31244e9d54da2b515c612c3b9c650)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:53.682914 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf14f2-9fab-4032-8bcd-f72903ddea05 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:53.736127 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94cefa-e487-4e55-bbe9-2208e31e55d5 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:53.738872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ae94cefa-e487-4e55-bbe9-2208e31e55d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:53.755817 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94cefa-e487-4e55-bbe9-2208e31e55d5 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:53.760547 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:53.782324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-63bf14f2-9fab-4032-8bcd-f72903ddea05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26e31244e9d54da2b515c612c3b9c650', 'request_id': 'req-63bf14f2-9fab-4032-8bcd-f72903ddea05'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.799366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-63bf14f2-9fab-4032-8bcd-f72903ddea05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26e31244e9d54da2b515c612c3b9c650', 'request_id': 'req-63bf14f2-9fab-4032-8bcd-f72903ddea05'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.819543 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-63bf14f2-9fab-4032-8bcd-f72903ddea05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26e31244e9d54da2b515c612c3b9c650', 'request_id': 'req-63bf14f2-9fab-4032-8bcd-f72903ddea05'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.856747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd7f14-c73e-4154-8c80-5b55e1d008ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:53.857768 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd7f14-c73e-4154-8c80-5b55e1d008ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:53.864278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd7f14-c73e-4154-8c80-5b55e1d008ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:53.864278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd7f14-c73e-4154-8c80-5b55e1d008ec admin admin] PATH_INFO: `/v3/projects/c1af57535ad042c490c73c2aeeced41c` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:53.864278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd7f14-c73e-4154-8c80-5b55e1d008ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1af57535ad042c490c73c2aeeced41c)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:53.864677 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-63bf14f2-9fab-4032-8bcd-f72903ddea05 admin admin] The token cache is being invalidate because project 26e31244e9d54da2b515c612c3b9c650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:53.864677 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-63bf14f2-9fab-4032-8bcd-f72903ddea05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.864677 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-63bf14f2-9fab-4032-8bcd-f72903ddea05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.864882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd7f14-c73e-4154-8c80-5b55e1d008ec admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:53.864955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 937/1865] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:53 2022] DELETE /identity/v3/projects/26e31244e9d54da2b515c612c3b9c650 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:53.867963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf14f2-9fab-4032-8bcd-f72903ddea05 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:53.869825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-63bf14f2-9fab-4032-8bcd-f72903ddea05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:53.876310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:53.886187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf14f2-9fab-4032-8bcd-f72903ddea05 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:53.891501 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adce7f3-22f5-485d-8340-87953aa1c740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:53.892050 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adce7f3-22f5-485d-8340-87953aa1c740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:53.892400 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adce7f3-22f5-485d-8340-87953aa1c740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:53.892668 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adce7f3-22f5-485d-8340-87953aa1c740 admin admin] PATH_INFO: `/v3/users/d33c436faebe4d0e9012ca89d42a895b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:53.893837 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adce7f3-22f5-485d-8340-87953aa1c740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d33c436faebe4d0e9012ca89d42a895b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:53.896292 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adce7f3-22f5-485d-8340-87953aa1c740 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:53.933762 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5fbd7f14-c73e-4154-8c80-5b55e1d008ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1af57535ad042c490c73c2aeeced41c', 'request_id': 'req-5fbd7f14-c73e-4154-8c80-5b55e1d008ec'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.944791 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5fbd7f14-c73e-4154-8c80-5b55e1d008ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1af57535ad042c490c73c2aeeced41c', 'request_id': 'req-5fbd7f14-c73e-4154-8c80-5b55e1d008ec'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.954651 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5fbd7f14-c73e-4154-8c80-5b55e1d008ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1af57535ad042c490c73c2aeeced41c', 'request_id': 'req-5fbd7f14-c73e-4154-8c80-5b55e1d008ec'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.966275 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5adce7f3-22f5-485d-8340-87953aa1c740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd33c436faebe4d0e9012ca89d42a895b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.974459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5fbd7f14-c73e-4154-8c80-5b55e1d008ec admin admin] The token cache is being invalidate because project c1af57535ad042c490c73c2aeeced41c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:53.974656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5fbd7f14-c73e-4154-8c80-5b55e1d008ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.975918 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5fbd7f14-c73e-4154-8c80-5b55e1d008ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.979199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 929/1866] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:53 2022] DELETE /identity/v3/projects/c1af57535ad042c490c73c2aeeced41c => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:53.983674 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5adce7f3-22f5-485d-8340-87953aa1c740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd33c436faebe4d0e9012ca89d42a895b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.985168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8f6bc-b888-4549-a650-b9b574b1a090 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:53.986074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8f6bc-b888-4549-a650-b9b574b1a090 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:53.986513 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8f6bc-b888-4549-a650-b9b574b1a090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:53.995581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5adce7f3-22f5-485d-8340-87953aa1c740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd33c436faebe4d0e9012ca89d42a895b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:53.996407 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5adce7f3-22f5-485d-8340-87953aa1c740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd33c436faebe4d0e9012ca89d42a895b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:54.009924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5adce7f3-22f5-485d-8340-87953aa1c740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd33c436faebe4d0e9012ca89d42a895b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:54.016878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-45e8f6bc-b888-4549-a650-b9b574b1a090 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:54.018625 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-45e8f6bc-b888-4549-a650-b9b574b1a090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:54.020054 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 938/1867] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:53 2022] DELETE /identity/v3/users/d33c436faebe4d0e9012ca89d42a895b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:54.139649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 930/1868] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:54.149372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93107167-3753-4cc4-97d8-10c15be712ed None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:54.149815 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93107167-3753-4cc4-97d8-10c15be712ed None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:54.150480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93107167-3753-4cc4-97d8-10c15be712ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:54.178320 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-93107167-3753-4cc4-97d8-10c15be712ed None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:54.180111 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-93107167-3753-4cc4-97d8-10c15be712ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:54.190595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 939/1869] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:55:54.196940 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8f6bc-b888-4549-a650-b9b574b1a090 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:54.209961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8f6bc-b888-4549-a650-b9b574b1a090 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:54.216524 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fc23e-fd78-4dd8-b9ae-cf283c42acff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:54.217379 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fc23e-fd78-4dd8-b9ae-cf283c42acff admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:54.217968 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fc23e-fd78-4dd8-b9ae-cf283c42acff admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:54.218403 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fc23e-fd78-4dd8-b9ae-cf283c42acff admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:54.219762 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94fc23e-fd78-4dd8-b9ae-cf283c42acff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:54.220508 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94fc23e-fd78-4dd8-b9ae-cf283c42acff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:54.222492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94fc23e-fd78-4dd8-b9ae-cf283c42acff admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:54.264628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 931/1870] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:54.267641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93107167-3753-4cc4-97d8-10c15be712ed None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:54.288312 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93107167-3753-4cc4-97d8-10c15be712ed None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:54.296337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263cee5-d712-4484-af37-ef588f3d1f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:54.297795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263cee5-d712-4484-af37-ef588f3d1f8c admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:54.298285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263cee5-d712-4484-af37-ef588f3d1f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:54.298989 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263cee5-d712-4484-af37-ef588f3d1f8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:54.300009 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263cee5-d712-4484-af37-ef588f3d1f8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:54.302444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263cee5-d712-4484-af37-ef588f3d1f8c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:54.339647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 940/1871] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:54 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:54.344577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fc23e-fd78-4dd8-b9ae-cf283c42acff admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:54.356923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fc23e-fd78-4dd8-b9ae-cf283c42acff admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:54.491009 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aabd68-1768-41ab-8b30-f561b344c2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:54.491650 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aabd68-1768-41ab-8b30-f561b344c2ea admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:54.492578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aabd68-1768-41ab-8b30-f561b344c2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:54.492797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aabd68-1768-41ab-8b30-f561b344c2ea admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:54.493589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aabd68-1768-41ab-8b30-f561b344c2ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:54.495543 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aabd68-1768-41ab-8b30-f561b344c2ea admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:54.549493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 932/1872] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:54 2022] POST /identity/v3/users => generated 292 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:54.554156 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263cee5-d712-4484-af37-ef588f3d1f8c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:54.574845 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263cee5-d712-4484-af37-ef588f3d1f8c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:54.581863 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb1d63-ab03-461e-aeac-7e840238444d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:54.582495 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb1d63-ab03-461e-aeac-7e840238444d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:54.582793 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb1d63-ab03-461e-aeac-7e840238444d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:54.583008 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb1d63-ab03-461e-aeac-7e840238444d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:54.583640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb1d63-ab03-461e-aeac-7e840238444d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:54.585273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb1d63-ab03-461e-aeac-7e840238444d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:54.610842 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 941/1873] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:54.614856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aabd68-1768-41ab-8b30-f561b344c2ea admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:54.623894 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb1d63-ab03-461e-aeac-7e840238444d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:54.629317 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-10bb1d63-ab03-461e-aeac-7e840238444d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:54.629470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aabd68-1768-41ab-8b30-f561b344c2ea admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:54.636124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88b11f-8da0-4bea-b69d-b09472f8c31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:54.636796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88b11f-8da0-4bea-b69d-b09472f8c31b admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:54.637131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88b11f-8da0-4bea-b69d-b09472f8c31b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:54.637458 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88b11f-8da0-4bea-b69d-b09472f8c31b admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:54.638135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf88b11f-8da0-4bea-b69d-b09472f8c31b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:54.640279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf88b11f-8da0-4bea-b69d-b09472f8c31b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:54.654104 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb1d63-ab03-461e-aeac-7e840238444d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:54.659027 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 933/1874] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:54.660987 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88b11f-8da0-4bea-b69d-b09472f8c31b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:54.666679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a96536-e03f-4c6f-bb3b-b22031147628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:54.669263 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:54.670076 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a96536-e03f-4c6f-bb3b-b22031147628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:54.670594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a96536-e03f-4c6f-bb3b-b22031147628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:54.671165 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a96536-e03f-4c6f-bb3b-b22031147628 admin admin] PATH_INFO: `/v3/users/7bcb6f5305214a2ba5764f02c58f498e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:54.673631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a96536-e03f-4c6f-bb3b-b22031147628 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bcb6f5305214a2ba5764f02c58f498e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:54.675183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88b11f-8da0-4bea-b69d-b09472f8c31b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:54.678366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a96536-e03f-4c6f-bb3b-b22031147628 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:54.685239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bfca0-2b6f-423b-9c51-3f8559797e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:54.688120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bfca0-2b6f-423b-9c51-3f8559797e36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:54.688120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bfca0-2b6f-423b-9c51-3f8559797e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:54.688120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bfca0-2b6f-423b-9c51-3f8559797e36 admin admin] PATH_INFO: `/v3/projects/f434235af61d4d39876919d96219bb22/users/2aad55ba48e8413ebcda78146ebc6974/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:54.720880 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08bfca0-2b6f-423b-9c51-3f8559797e36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f434235af61d4d39876919d96219bb22, user_id=2aad55ba48e8413ebcda78146ebc6974, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:54.724389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08bfca0-2b6f-423b-9c51-3f8559797e36 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:54.756544 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 934/1875] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:54 2022] PUT /identity/v3/projects/f434235af61d4d39876919d96219bb22/users/2aad55ba48e8413ebcda78146ebc6974/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:54.769765 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c597008-26c6-4bab-bf84-cfd1a87ab83b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:54.770169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c597008-26c6-4bab-bf84-cfd1a87ab83b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:54.770602 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c597008-26c6-4bab-bf84-cfd1a87ab83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:54.798498 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-02a96536-e03f-4c6f-bb3b-b22031147628 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bcb6f5305214a2ba5764f02c58f498e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:54.818241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=75,keystone:DELETE=22,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:54.821276 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-02a96536-e03f-4c6f-bb3b-b22031147628 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bcb6f5305214a2ba5764f02c58f498e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:54.841297 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1c597008-26c6-4bab-bf84-cfd1a87ab83b None None] MFA Rules not processed for user `2aad55ba48e8413ebcda78146ebc6974`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:54.842679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1c597008-26c6-4bab-bf84-cfd1a87ab83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:54.848681 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-02a96536-e03f-4c6f-bb3b-b22031147628 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bcb6f5305214a2ba5764f02c58f498e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:54.849683 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-02a96536-e03f-4c6f-bb3b-b22031147628 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bcb6f5305214a2ba5764f02c58f498e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:54.885012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-02a96536-e03f-4c6f-bb3b-b22031147628 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bcb6f5305214a2ba5764f02c58f498e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:54.922437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 942/1876] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:54 2022] DELETE /identity/v3/users/7bcb6f5305214a2ba5764f02c58f498e => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:54.922437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a96536-e03f-4c6f-bb3b-b22031147628 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:54.922437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-02a96536-e03f-4c6f-bb3b-b22031147628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:54.967479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 935/1877] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:54.967479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b3345-7895-436c-baf7-7b7468e84c07 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:54.967479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b3345-7895-436c-baf7-7b7468e84c07 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:54.967479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b3345-7895-436c-baf7-7b7468e84c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:54.999166 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-db5b3345-7895-436c-baf7-7b7468e84c07 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:55.000544 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-db5b3345-7895-436c-baf7-7b7468e84c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:55.049624 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a96536-e03f-4c6f-bb3b-b22031147628 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:55.063403 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98b471-0802-41ed-b121-dc675d3f8ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:55.064064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98b471-0802-41ed-b121-dc675d3f8ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:55.064369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98b471-0802-41ed-b121-dc675d3f8ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:55.065208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98b471-0802-41ed-b121-dc675d3f8ca5 admin admin] PATH_INFO: `/v3/projects/9c3f5e5d2bde492cb89fd66b5901b074` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:55.068735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98b471-0802-41ed-b121-dc675d3f8ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c3f5e5d2bde492cb89fd66b5901b074)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:55.072302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98b471-0802-41ed-b121-dc675d3f8ca5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:55.095964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 936/1878] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:55.096988 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b3345-7895-436c-baf7-7b7468e84c07 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:55.098739 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-db5b3345-7895-436c-baf7-7b7468e84c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:55.136856 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cf98b471-0802-41ed-b121-dc675d3f8ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c3f5e5d2bde492cb89fd66b5901b074', 'request_id': 'req-cf98b471-0802-41ed-b121-dc675d3f8ca5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:55.149021 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cf98b471-0802-41ed-b121-dc675d3f8ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c3f5e5d2bde492cb89fd66b5901b074', 'request_id': 'req-cf98b471-0802-41ed-b121-dc675d3f8ca5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:55.157654 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cf98b471-0802-41ed-b121-dc675d3f8ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c3f5e5d2bde492cb89fd66b5901b074', 'request_id': 'req-cf98b471-0802-41ed-b121-dc675d3f8ca5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:55.172125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b3345-7895-436c-baf7-7b7468e84c07 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:55.173638 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cf98b471-0802-41ed-b121-dc675d3f8ca5 admin admin] The token cache is being invalidate because project 9c3f5e5d2bde492cb89fd66b5901b074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:55.174004 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cf98b471-0802-41ed-b121-dc675d3f8ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:55.174730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cf98b471-0802-41ed-b121-dc675d3f8ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:55.176665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 943/1879] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:54 2022] DELETE /identity/v3/projects/9c3f5e5d2bde492cb89fd66b5901b074 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:55.177795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98b471-0802-41ed-b121-dc675d3f8ca5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:55.179225 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cf98b471-0802-41ed-b121-dc675d3f8ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:55.203479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:55.237483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b9fab-b935-45c6-940f-79f717094cf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:55.238183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b9fab-b935-45c6-940f-79f717094cf3 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:55.238525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b9fab-b935-45c6-940f-79f717094cf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:55.238952 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b9fab-b935-45c6-940f-79f717094cf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:55.241524 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-846b9fab-b935-45c6-940f-79f717094cf3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:55.246165 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98b471-0802-41ed-b121-dc675d3f8ca5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:55.250002 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13444322-c039-45bd-b012-ed24db2bb3d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:55.250713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13444322-c039-45bd-b012-ed24db2bb3d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:55.251280 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13444322-c039-45bd-b012-ed24db2bb3d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:55.251724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13444322-c039-45bd-b012-ed24db2bb3d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:55.253501 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-13444322-c039-45bd-b012-ed24db2bb3d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:55.289293 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:55.327680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b9fab-b935-45c6-940f-79f717094cf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:55.329237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b9fab-b935-45c6-940f-79f717094cf3 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:55.340613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13444322-c039-45bd-b012-ed24db2bb3d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:55.342272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13444322-c039-45bd-b012-ed24db2bb3d9 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:55.344177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 937/1880] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:55.345493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac54c08-dfab-4a8e-93d2-7967626674c8 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:55.345607 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac54c08-dfab-4a8e-93d2-7967626674c8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:55.345977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac54c08-dfab-4a8e-93d2-7967626674c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:55.356580 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 944/1881] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:55.367174 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-dac54c08-dfab-4a8e-93d2-7967626674c8 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:55.368461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dac54c08-dfab-4a8e-93d2-7967626674c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:55.377395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13444322-c039-45bd-b012-ed24db2bb3d9 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:55.379409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-13444322-c039-45bd-b012-ed24db2bb3d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:55.384320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 938/1882] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:55.394453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b9634-fa30-41a9-aa70-a3b28d037018 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:55.394804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b9634-fa30-41a9-aa70-a3b28d037018 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:55.395068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13444322-c039-45bd-b012-ed24db2bb3d9 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:55.395181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b9634-fa30-41a9-aa70-a3b28d037018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:55.404438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97cef2-70f1-4b34-9754-7cceb7f2059b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:55.406299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97cef2-70f1-4b34-9754-7cceb7f2059b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:55.407058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97cef2-70f1-4b34-9754-7cceb7f2059b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:55.407543 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97cef2-70f1-4b34-9754-7cceb7f2059b admin admin] PATH_INFO: `/v3/users/2aad55ba48e8413ebcda78146ebc6974` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:55.410271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba97cef2-70f1-4b34-9754-7cceb7f2059b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aad55ba48e8413ebcda78146ebc6974)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:55.412997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba97cef2-70f1-4b34-9754-7cceb7f2059b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:55.423082 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-280b9634-fa30-41a9-aa70-a3b28d037018 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:55.424381 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-280b9634-fa30-41a9-aa70-a3b28d037018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:55.438668 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 939/1883] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:55.443053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b9634-fa30-41a9-aa70-a3b28d037018 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:55.458606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b9634-fa30-41a9-aa70-a3b28d037018 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:55.466288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142fddb8-1258-4a17-a449-67fa090de921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:55.467118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fddb8-1258-4a17-a449-67fa090de921 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:55.467476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fddb8-1258-4a17-a449-67fa090de921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:55.467791 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fddb8-1258-4a17-a449-67fa090de921 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:55.468571 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142fddb8-1258-4a17-a449-67fa090de921 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:55.469013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142fddb8-1258-4a17-a449-67fa090de921 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:55.471262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142fddb8-1258-4a17-a449-67fa090de921 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:55.482146 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ba97cef2-70f1-4b34-9754-7cceb7f2059b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aad55ba48e8413ebcda78146ebc6974'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:55.491551 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 940/1884] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:55.492706 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142fddb8-1258-4a17-a449-67fa090de921 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:55.501927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ba97cef2-70f1-4b34-9754-7cceb7f2059b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aad55ba48e8413ebcda78146ebc6974'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:55.505070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142fddb8-1258-4a17-a449-67fa090de921 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:55.512564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0360dd7-1a1c-424e-84c6-141c84de8301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:55.512724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ba97cef2-70f1-4b34-9754-7cceb7f2059b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aad55ba48e8413ebcda78146ebc6974'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:55.513174 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0360dd7-1a1c-424e-84c6-141c84de8301 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:55.513577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0360dd7-1a1c-424e-84c6-141c84de8301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:55.513855 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0360dd7-1a1c-424e-84c6-141c84de8301 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:55.514091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ba97cef2-70f1-4b34-9754-7cceb7f2059b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aad55ba48e8413ebcda78146ebc6974'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:55.514493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0360dd7-1a1c-424e-84c6-141c84de8301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:55.517224 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0360dd7-1a1c-424e-84c6-141c84de8301 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:55.524806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ba97cef2-70f1-4b34-9754-7cceb7f2059b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aad55ba48e8413ebcda78146ebc6974'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:55.536539 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 945/1885] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:55 2022] DELETE /identity/v3/users/2aad55ba48e8413ebcda78146ebc6974 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:55.542176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83208e-a79f-4547-99ff-f6b0c3a15104 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:55.542521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83208e-a79f-4547-99ff-f6b0c3a15104 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:55.542788 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83208e-a79f-4547-99ff-f6b0c3a15104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:55.579412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 941/1886] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:55 2022] POST /identity/v3/projects => generated 383 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:55.579568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1e83208e-a79f-4547-99ff-f6b0c3a15104 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:55.581007 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1e83208e-a79f-4547-99ff-f6b0c3a15104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:55.584273 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0360dd7-1a1c-424e-84c6-141c84de8301 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:55.590286 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e0360dd7-1a1c-424e-84c6-141c84de8301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:55.688664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 946/1887] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:55:55.729465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0360dd7-1a1c-424e-84c6-141c84de8301 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:55.734814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bb91c-3e9a-446d-94c3-7f4bc944ca10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:55.735509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bb91c-3e9a-446d-94c3-7f4bc944ca10 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:55.735890 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bb91c-3e9a-446d-94c3-7f4bc944ca10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:55.736955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bb91c-3e9a-446d-94c3-7f4bc944ca10 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:55.737613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49bb91c-3e9a-446d-94c3-7f4bc944ca10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:55.740361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49bb91c-3e9a-446d-94c3-7f4bc944ca10 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:55.743000 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83208e-a79f-4547-99ff-f6b0c3a15104 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:55.744626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1e83208e-a79f-4547-99ff-f6b0c3a15104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:55.790396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 942/1888] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:55 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:55.795259 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bb91c-3e9a-446d-94c3-7f4bc944ca10 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:55.814618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bb91c-3e9a-446d-94c3-7f4bc944ca10 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:55.822512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773d23e-6250-41e4-8a13-3b6024db32cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:55.823128 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773d23e-6250-41e4-8a13-3b6024db32cf admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:55.823407 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773d23e-6250-41e4-8a13-3b6024db32cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:55.823679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773d23e-6250-41e4-8a13-3b6024db32cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:55.824948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4773d23e-6250-41e4-8a13-3b6024db32cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:55.830500 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4773d23e-6250-41e4-8a13-3b6024db32cf admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:55.846929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 943/1889] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:55:55.852479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773d23e-6250-41e4-8a13-3b6024db32cf admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:55.852751 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:55.868431 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773d23e-6250-41e4-8a13-3b6024db32cf admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:55.876682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5aa541-efe9-4c76-92b5-129d85b369c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:55.877439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5aa541-efe9-4c76-92b5-129d85b369c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:55.877813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5aa541-efe9-4c76-92b5-129d85b369c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:55.878553 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5aa541-efe9-4c76-92b5-129d85b369c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:55.879297 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5aa541-efe9-4c76-92b5-129d85b369c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:55.881499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5aa541-efe9-4c76-92b5-129d85b369c4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:55.889852 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:55.902665 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 944/1890] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:55.906767 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5aa541-efe9-4c76-92b5-129d85b369c4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:55.908664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83208e-a79f-4547-99ff-f6b0c3a15104 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:55.916062 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17213c92-b605-465d-b182-a455d9304514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:55.916814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17213c92-b605-465d-b182-a455d9304514 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:55.917045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17213c92-b605-465d-b182-a455d9304514 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:55.917379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17213c92-b605-465d-b182-a455d9304514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:55.922433 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5aa541-efe9-4c76-92b5-129d85b369c4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:55.930751 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6449ac-e299-4c5b-9342-b115afa04397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:55.931988 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6449ac-e299-4c5b-9342-b115afa04397 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:55.932385 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6449ac-e299-4c5b-9342-b115afa04397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:55.932875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6449ac-e299-4c5b-9342-b115afa04397 admin admin] PATH_INFO: `/v3/projects/e8fd78eed2114eb7a2140f609993ff82/users/74d50ad0f0ad47238d47e5ce5bab9c10/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:55.942018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17213c92-b605-465d-b182-a455d9304514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:55.944740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17213c92-b605-465d-b182-a455d9304514 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:55.958752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 947/1891] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 07:55:55.987084 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6449ac-e299-4c5b-9342-b115afa04397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8fd78eed2114eb7a2140f609993ff82, user_id=74d50ad0f0ad47238d47e5ce5bab9c10, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:55.995278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6449ac-e299-4c5b-9342-b115afa04397 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:56.021071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 945/1892] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:55 2022] PUT /identity/v3/projects/e8fd78eed2114eb7a2140f609993ff82/users/74d50ad0f0ad47238d47e5ce5bab9c10/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:56.032577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5876d-2869-412d-9ab7-d4bdf6d51eb1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:56.032862 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5876d-2869-412d-9ab7-d4bdf6d51eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:56.033695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5876d-2869-412d-9ab7-d4bdf6d51eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:56.077190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a7d5876d-2869-412d-9ab7-d4bdf6d51eb1 None None] MFA Rules not processed for user `74d50ad0f0ad47238d47e5ce5bab9c10`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:56.078604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a7d5876d-2869-412d-9ab7-d4bdf6d51eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:56.158571 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 948/1893] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:56.163254 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6449ac-e299-4c5b-9342-b115afa04397 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:56.241554 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6449ac-e299-4c5b-9342-b115afa04397 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:56.249794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14a5d7-d588-409d-81d7-6e2c33715c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:56.249794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14a5d7-d588-409d-81d7-6e2c33715c1e admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:56.249794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14a5d7-d588-409d-81d7-6e2c33715c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:56.250178 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14a5d7-d588-409d-81d7-6e2c33715c1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:56.250838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14a5d7-d588-409d-81d7-6e2c33715c1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:56.253979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14a5d7-d588-409d-81d7-6e2c33715c1e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:56.293504 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 946/1894] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:56 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:56.294751 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5876d-2869-412d-9ab7-d4bdf6d51eb1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:56.310058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5876d-2869-412d-9ab7-d4bdf6d51eb1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:56.399260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab48a9-c8ef-4085-b8bb-0c155af9aa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:56.400186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab48a9-c8ef-4085-b8bb-0c155af9aa8e admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:56.401343 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab48a9-c8ef-4085-b8bb-0c155af9aa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:56.401965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab48a9-c8ef-4085-b8bb-0c155af9aa8e admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:56.403372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab48a9-c8ef-4085-b8bb-0c155af9aa8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:56.407051 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab48a9-c8ef-4085-b8bb-0c155af9aa8e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:56.438520 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 949/1895] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:56 2022] POST /identity/v3/users => generated 296 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:56.443333 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14a5d7-d588-409d-81d7-6e2c33715c1e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:56.457109 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14a5d7-d588-409d-81d7-6e2c33715c1e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:56.464469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d6aa3-f8bd-4eaa-9886-fac93a16db3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:56.465092 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d6aa3-f8bd-4eaa-9886-fac93a16db3f admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:56.465369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d6aa3-f8bd-4eaa-9886-fac93a16db3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:56.465643 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d6aa3-f8bd-4eaa-9886-fac93a16db3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:56.466463 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d6aa3-f8bd-4eaa-9886-fac93a16db3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:56.468430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d6aa3-f8bd-4eaa-9886-fac93a16db3f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:56.484001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 947/1896] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:55:56.488025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab48a9-c8ef-4085-b8bb-0c155af9aa8e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:56.501055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab48a9-c8ef-4085-b8bb-0c155af9aa8e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:56.506981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33fd5eb-931f-4c6e-a507-1d5df7794496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:56.508216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fd5eb-931f-4c6e-a507-1d5df7794496 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:56.508556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fd5eb-931f-4c6e-a507-1d5df7794496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:56.508793 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fd5eb-931f-4c6e-a507-1d5df7794496 admin admin] PATH_INFO: `/v3/projects/3d9edb804c804c07a89f17c302a6b8b3/users/cafecd37bb104cba934d51608a0178cc/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:56.541059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33fd5eb-931f-4c6e-a507-1d5df7794496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d9edb804c804c07a89f17c302a6b8b3, user_id=cafecd37bb104cba934d51608a0178cc, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:56.546505 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33fd5eb-931f-4c6e-a507-1d5df7794496 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:56.582076 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 950/1897] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:56 2022] PUT /identity/v3/projects/3d9edb804c804c07a89f17c302a6b8b3/users/cafecd37bb104cba934d51608a0178cc/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:56.586615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d6aa3-f8bd-4eaa-9886-fac93a16db3f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:56.616613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d6aa3-f8bd-4eaa-9886-fac93a16db3f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:56.733724 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58662e-d412-4e1a-b579-519451e24dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:56.734432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58662e-d412-4e1a-b579-519451e24dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:56.734700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58662e-d412-4e1a-b579-519451e24dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:56.734975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58662e-d412-4e1a-b579-519451e24dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:56.737936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58662e-d412-4e1a-b579-519451e24dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:56.740134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58662e-d412-4e1a-b579-519451e24dc5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:56.756845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 948/1898] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:56.760625 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33fd5eb-931f-4c6e-a507-1d5df7794496 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:56.775120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33fd5eb-931f-4c6e-a507-1d5df7794496 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:56.780102 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256711b-0254-451b-8292-9764f645f525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:56.780854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256711b-0254-451b-8292-9764f645f525 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:56.781172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256711b-0254-451b-8292-9764f645f525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:56.781451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256711b-0254-451b-8292-9764f645f525 admin admin] PATH_INFO: `/v3/projects/3d9edb804c804c07a89f17c302a6b8b3/users/cafecd37bb104cba934d51608a0178cc/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:56.783446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256711b-0254-451b-8292-9764f645f525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d9edb804c804c07a89f17c302a6b8b3, user_id=cafecd37bb104cba934d51608a0178cc, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:56.786160 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256711b-0254-451b-8292-9764f645f525 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:56.799645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 951/1899] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:56 2022] PUT /identity/v3/projects/3d9edb804c804c07a89f17c302a6b8b3/users/cafecd37bb104cba934d51608a0178cc/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:56.812958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e555dbc-f616-4d8d-bb8c-e0e317e04542 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:56.813469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e555dbc-f616-4d8d-bb8c-e0e317e04542 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:56.813469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e555dbc-f616-4d8d-bb8c-e0e317e04542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:56.862924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-0e555dbc-f616-4d8d-bb8c-e0e317e04542 None None] MFA Rules not processed for user `cafecd37bb104cba934d51608a0178cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:56.864267 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0e555dbc-f616-4d8d-bb8c-e0e317e04542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:56.915621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:56.987038 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 949/1900] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:57.002693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256711b-0254-451b-8292-9764f645f525 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:57.010301 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e555dbc-f616-4d8d-bb8c-e0e317e04542 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:57.012834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0e555dbc-f616-4d8d-bb8c-e0e317e04542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:57.015248 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9256711b-0254-451b-8292-9764f645f525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:57.059706 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:57.112910 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256711b-0254-451b-8292-9764f645f525 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:57.118966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd205d8-be12-4901-8af2-6c2fef5f8bc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:57.119853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd205d8-be12-4901-8af2-6c2fef5f8bc0 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:57.120269 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd205d8-be12-4901-8af2-6c2fef5f8bc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:57.121364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd205d8-be12-4901-8af2-6c2fef5f8bc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:57.134564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd205d8-be12-4901-8af2-6c2fef5f8bc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:57.136191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd205d8-be12-4901-8af2-6c2fef5f8bc0 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:57.152608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 952/1901] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:57.167981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e555dbc-f616-4d8d-bb8c-e0e317e04542 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:57.186563 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbafa5-292a-45c5-89e0-02b3e429e49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:57.187235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbafa5-292a-45c5-89e0-02b3e429e49f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:57.187487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbafa5-292a-45c5-89e0-02b3e429e49f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:57.187761 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbafa5-292a-45c5-89e0-02b3e429e49f admin admin] PATH_INFO: `/v3/projects/58f70c9b205b46debdb421351bf32ef6` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:57.190374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defbafa5-292a-45c5-89e0-02b3e429e49f admin admin] RBAC: Authorizing `identity:delete_project(project_id=58f70c9b205b46debdb421351bf32ef6)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:57.192913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defbafa5-292a-45c5-89e0-02b3e429e49f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:57.268992 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-defbafa5-292a-45c5-89e0-02b3e429e49f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58f70c9b205b46debdb421351bf32ef6', 'request_id': 'req-defbafa5-292a-45c5-89e0-02b3e429e49f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:57.276225 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-defbafa5-292a-45c5-89e0-02b3e429e49f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58f70c9b205b46debdb421351bf32ef6', 'request_id': 'req-defbafa5-292a-45c5-89e0-02b3e429e49f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:57.287807 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-defbafa5-292a-45c5-89e0-02b3e429e49f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58f70c9b205b46debdb421351bf32ef6', 'request_id': 'req-defbafa5-292a-45c5-89e0-02b3e429e49f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:57.317051 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-defbafa5-292a-45c5-89e0-02b3e429e49f admin admin] The token cache is being invalidate because project 58f70c9b205b46debdb421351bf32ef6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:57.317377 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-defbafa5-292a-45c5-89e0-02b3e429e49f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:57.320936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-defbafa5-292a-45c5-89e0-02b3e429e49f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:57.323620 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 950/1902] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:57 2022] DELETE /identity/v3/projects/58f70c9b205b46debdb421351bf32ef6 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:57.327629 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd205d8-be12-4901-8af2-6c2fef5f8bc0 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:57.334611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbafa5-292a-45c5-89e0-02b3e429e49f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:57.337097 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd205d8-be12-4901-8af2-6c2fef5f8bc0 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:57.339439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-defbafa5-292a-45c5-89e0-02b3e429e49f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:57.474488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82938db-1bc8-4298-a5c2-431c980359ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:57.476957 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82938db-1bc8-4298-a5c2-431c980359ba service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:57.477508 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82938db-1bc8-4298-a5c2-431c980359ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:57.477886 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82938db-1bc8-4298-a5c2-431c980359ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:57.482576 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbafa5-292a-45c5-89e0-02b3e429e49f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:57.483375 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e82938db-1bc8-4298-a5c2-431c980359ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:57.496297 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79df0e4-ddf1-45f8-8d14-bc7746ce75d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:57.497138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79df0e4-ddf1-45f8-8d14-bc7746ce75d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:57.497416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79df0e4-ddf1-45f8-8d14-bc7746ce75d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:57.497684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79df0e4-ddf1-45f8-8d14-bc7746ce75d0 admin admin] PATH_INFO: `/v3/users/5f42307bfccc4a86b35ae60a1c6f56c1` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:57.499699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79df0e4-ddf1-45f8-8d14-bc7746ce75d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f42307bfccc4a86b35ae60a1c6f56c1)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:57.502102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79df0e4-ddf1-45f8-8d14-bc7746ce75d0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:57.505486 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:57.625814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82938db-1bc8-4298-a5c2-431c980359ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:57.627480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82938db-1bc8-4298-a5c2-431c980359ba service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:57.634484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d79df0e4-ddf1-45f8-8d14-bc7746ce75d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f42307bfccc4a86b35ae60a1c6f56c1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:57.636364 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d79df0e4-ddf1-45f8-8d14-bc7746ce75d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f42307bfccc4a86b35ae60a1c6f56c1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:57.646200 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 953/1903] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:57.647313 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82938db-1bc8-4298-a5c2-431c980359ba service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:57.649138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e82938db-1bc8-4298-a5c2-431c980359ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:57.673061 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d79df0e4-ddf1-45f8-8d14-bc7746ce75d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f42307bfccc4a86b35ae60a1c6f56c1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:57.674263 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d79df0e4-ddf1-45f8-8d14-bc7746ce75d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f42307bfccc4a86b35ae60a1c6f56c1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:57.674478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82938db-1bc8-4298-a5c2-431c980359ba service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:57.687198 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86689a-105b-4bb4-ab7e-8031bb8bcf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:57.688551 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86689a-105b-4bb4-ab7e-8031bb8bcf7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:57.689264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86689a-105b-4bb4-ab7e-8031bb8bcf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:57.689907 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86689a-105b-4bb4-ab7e-8031bb8bcf7a admin admin] PATH_INFO: `/v3/projects/f434235af61d4d39876919d96219bb22` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:57.691352 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86689a-105b-4bb4-ab7e-8031bb8bcf7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f434235af61d4d39876919d96219bb22)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:57.695641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86689a-105b-4bb4-ab7e-8031bb8bcf7a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:57.713761 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d79df0e4-ddf1-45f8-8d14-bc7746ce75d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f42307bfccc4a86b35ae60a1c6f56c1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:57.751583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 951/1904] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:57 2022] DELETE /identity/v3/users/5f42307bfccc4a86b35ae60a1c6f56c1 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:57.791627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0e86689a-105b-4bb4-ab7e-8031bb8bcf7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f434235af61d4d39876919d96219bb22', 'request_id': 'req-0e86689a-105b-4bb4-ab7e-8031bb8bcf7a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:57.801430 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0e86689a-105b-4bb4-ab7e-8031bb8bcf7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f434235af61d4d39876919d96219bb22', 'request_id': 'req-0e86689a-105b-4bb4-ab7e-8031bb8bcf7a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:57.830427 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0e86689a-105b-4bb4-ab7e-8031bb8bcf7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f434235af61d4d39876919d96219bb22', 'request_id': 'req-0e86689a-105b-4bb4-ab7e-8031bb8bcf7a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:57.862270 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:57.890659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0e86689a-105b-4bb4-ab7e-8031bb8bcf7a admin admin] The token cache is being invalidate because project f434235af61d4d39876919d96219bb22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:57.890966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0e86689a-105b-4bb4-ab7e-8031bb8bcf7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:57.893018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0e86689a-105b-4bb4-ab7e-8031bb8bcf7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:57.899733 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 954/1905] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:57 2022] DELETE /identity/v3/projects/f434235af61d4d39876919d96219bb22 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:57.902092 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def45833-b50e-460b-a692-5ad2e0b6eb14 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:57.903147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def45833-b50e-460b-a692-5ad2e0b6eb14 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:57.903651 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def45833-b50e-460b-a692-5ad2e0b6eb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:57.926930 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-def45833-b50e-460b-a692-5ad2e0b6eb14 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:57.928580 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-def45833-b50e-460b-a692-5ad2e0b6eb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:58.042989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 952/1906] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:58.050694 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa6c50-034c-4e67-b643-6952dd3c8b77 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:58.051507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa6c50-034c-4e67-b643-6952dd3c8b77 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:58.051947 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa6c50-034c-4e67-b643-6952dd3c8b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:58.088056 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6eaa6c50-034c-4e67-b643-6952dd3c8b77 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:58.089917 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6eaa6c50-034c-4e67-b643-6952dd3c8b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:58.099322 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 955/1907] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:58.104312 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def45833-b50e-460b-a692-5ad2e0b6eb14 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:58.118693 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def45833-b50e-460b-a692-5ad2e0b6eb14 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:58.124592 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a6ccf-9b55-438a-a87c-9580062ed8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:58.125218 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a6ccf-9b55-438a-a87c-9580062ed8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:58.125494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a6ccf-9b55-438a-a87c-9580062ed8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:58.125762 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a6ccf-9b55-438a-a87c-9580062ed8f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:58.126580 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a6ccf-9b55-438a-a87c-9580062ed8f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:58.126882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a6ccf-9b55-438a-a87c-9580062ed8f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:58.129031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a6ccf-9b55-438a-a87c-9580062ed8f0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:58.139619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa6c50-034c-4e67-b643-6952dd3c8b77 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:58.141299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6eaa6c50-034c-4e67-b643-6952dd3c8b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:58.166067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 953/1908] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:58.166572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa6c50-034c-4e67-b643-6952dd3c8b77 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:58.171101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a6ccf-9b55-438a-a87c-9580062ed8f0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:58.185807 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c9be1b-dd80-4141-823f-690a2b47f284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:58.186464 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9be1b-dd80-4141-823f-690a2b47f284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:58.186741 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9be1b-dd80-4141-823f-690a2b47f284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:58.187012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9be1b-dd80-4141-823f-690a2b47f284 admin admin] PATH_INFO: `/v3/users/74d50ad0f0ad47238d47e5ce5bab9c10` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:58.188267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9be1b-dd80-4141-823f-690a2b47f284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74d50ad0f0ad47238d47e5ce5bab9c10)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:58.190210 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9be1b-dd80-4141-823f-690a2b47f284 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:58.205986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a6ccf-9b55-438a-a87c-9580062ed8f0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:58.213422 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d062d3-889e-4190-9178-d5d054cea0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:58.214620 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d062d3-889e-4190-9178-d5d054cea0a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:58.214902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d062d3-889e-4190-9178-d5d054cea0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:58.215190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d062d3-889e-4190-9178-d5d054cea0a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:58.215858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d062d3-889e-4190-9178-d5d054cea0a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:58.218181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d062d3-889e-4190-9178-d5d054cea0a9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:58.295511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 954/1909] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:58 2022] POST /identity/v3/projects => generated 361 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:58.296441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d062d3-889e-4190-9178-d5d054cea0a9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:58.297184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-26c9be1b-dd80-4141-823f-690a2b47f284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74d50ad0f0ad47238d47e5ce5bab9c10'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:58.327435 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-26c9be1b-dd80-4141-823f-690a2b47f284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74d50ad0f0ad47238d47e5ce5bab9c10'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:58.342240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-26c9be1b-dd80-4141-823f-690a2b47f284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74d50ad0f0ad47238d47e5ce5bab9c10'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:58.343020 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-26c9be1b-dd80-4141-823f-690a2b47f284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74d50ad0f0ad47238d47e5ce5bab9c10'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:58.354981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-26c9be1b-dd80-4141-823f-690a2b47f284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74d50ad0f0ad47238d47e5ce5bab9c10'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:58.366887 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 956/1910] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:58 2022] DELETE /identity/v3/users/74d50ad0f0ad47238d47e5ce5bab9c10 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:55:58.371702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa03bd-f04a-4456-9606-194f3c983c29 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:58.372006 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa03bd-f04a-4456-9606-194f3c983c29 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:58.372286 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa03bd-f04a-4456-9606-194f3c983c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:58.395470 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9daa03bd-f04a-4456-9606-194f3c983c29 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:58.398896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9daa03bd-f04a-4456-9606-194f3c983c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:58.537189 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:58.563262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d062d3-889e-4190-9178-d5d054cea0a9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:58.564727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 957/1911] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:58.566306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa03bd-f04a-4456-9606-194f3c983c29 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:58.571661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aa2d5-dc33-464b-a5e8-da84e7265db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:58.572350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aa2d5-dc33-464b-a5e8-da84e7265db5 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:58.572627 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aa2d5-dc33-464b-a5e8-da84e7265db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:58.572912 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aa2d5-dc33-464b-a5e8-da84e7265db5 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:58.573537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0aa2d5-dc33-464b-a5e8-da84e7265db5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:58.575733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0aa2d5-dc33-464b-a5e8-da84e7265db5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:58.578899 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9daa03bd-f04a-4456-9606-194f3c983c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:58.594861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa03bd-f04a-4456-9606-194f3c983c29 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:58.603600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72371a73-6ad2-45a4-8b7b-f59805f770d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:58.604741 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72371a73-6ad2-45a4-8b7b-f59805f770d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:58.605307 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72371a73-6ad2-45a4-8b7b-f59805f770d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:58.605768 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72371a73-6ad2-45a4-8b7b-f59805f770d1 admin admin] PATH_INFO: `/v3/projects/91b3d41051a34a9a9a805fdfd0f469bf` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:58.607234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 955/1912] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:58 2022] POST /identity/v3/users => generated 281 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:58.608055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72371a73-6ad2-45a4-8b7b-f59805f770d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91b3d41051a34a9a9a805fdfd0f469bf)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:58.608195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aa2d5-dc33-464b-a5e8-da84e7265db5 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:58.610114 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5b0aa2d5-dc33-464b-a5e8-da84e7265db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:58.610841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72371a73-6ad2-45a4-8b7b-f59805f770d1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:58.713576 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aa2d5-dc33-464b-a5e8-da84e7265db5 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:58.725545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-72371a73-6ad2-45a4-8b7b-f59805f770d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91b3d41051a34a9a9a805fdfd0f469bf', 'request_id': 'req-72371a73-6ad2-45a4-8b7b-f59805f770d1'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:58.733645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-72371a73-6ad2-45a4-8b7b-f59805f770d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91b3d41051a34a9a9a805fdfd0f469bf', 'request_id': 'req-72371a73-6ad2-45a4-8b7b-f59805f770d1'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:58.734880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:58.750384 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-72371a73-6ad2-45a4-8b7b-f59805f770d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91b3d41051a34a9a9a805fdfd0f469bf', 'request_id': 'req-72371a73-6ad2-45a4-8b7b-f59805f770d1'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:58.769732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-72371a73-6ad2-45a4-8b7b-f59805f770d1 admin admin] The token cache is being invalidate because project 91b3d41051a34a9a9a805fdfd0f469bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:55:58.770130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-72371a73-6ad2-45a4-8b7b-f59805f770d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:58.780737 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-72371a73-6ad2-45a4-8b7b-f59805f770d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:58.782570 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 958/1913] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:58 2022] DELETE /identity/v3/projects/91b3d41051a34a9a9a805fdfd0f469bf => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:58.786324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72371a73-6ad2-45a4-8b7b-f59805f770d1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:58.787755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-72371a73-6ad2-45a4-8b7b-f59805f770d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:58.803234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa3563-00c7-41a7-a914-81ec99528ca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:58.804326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa3563-00c7-41a7-a914-81ec99528ca5 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:58.804826 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa3563-00c7-41a7-a914-81ec99528ca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:58.805107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa3563-00c7-41a7-a914-81ec99528ca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:58.808763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6ffa3563-00c7-41a7-a914-81ec99528ca5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:58.921025 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa3563-00c7-41a7-a914-81ec99528ca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:58.925294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa3563-00c7-41a7-a914-81ec99528ca5 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:58.937786 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72371a73-6ad2-45a4-8b7b-f59805f770d1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:58.950285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368eb13-d3dc-410c-aa0f-2bced9740fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:58.950684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368eb13-d3dc-410c-aa0f-2bced9740fff admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:58.951101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 956/1914] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:55:58.951228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368eb13-d3dc-410c-aa0f-2bced9740fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:58.951228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368eb13-d3dc-410c-aa0f-2bced9740fff admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:58.951869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368eb13-d3dc-410c-aa0f-2bced9740fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:58.953674 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368eb13-d3dc-410c-aa0f-2bced9740fff admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:58.958708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41af04-3620-4e1f-8242-8ee41e6eb2c8 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:58.958708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41af04-3620-4e1f-8242-8ee41e6eb2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:58.959071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41af04-3620-4e1f-8242-8ee41e6eb2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:58.975041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 959/1915] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:58.977403 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368eb13-d3dc-410c-aa0f-2bced9740fff admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:58.991577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368eb13-d3dc-410c-aa0f-2bced9740fff admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:58.997699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba7600-1112-4111-8799-6fd7f0181583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:58.998912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba7600-1112-4111-8799-6fd7f0181583 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:58.999831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba7600-1112-4111-8799-6fd7f0181583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:59.001280 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba7600-1112-4111-8799-6fd7f0181583 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:59.002231 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba7600-1112-4111-8799-6fd7f0181583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:59.004177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba7600-1112-4111-8799-6fd7f0181583 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:59.014003 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-cf41af04-3620-4e1f-8242-8ee41e6eb2c8 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:59.015234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cf41af04-3620-4e1f-8242-8ee41e6eb2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:59.035059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 960/1916] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:59.037210 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba7600-1112-4111-8799-6fd7f0181583 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:59.042414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 957/1917] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:59.047835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c3787-f2fc-45f3-bfde-cfaff4121123 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:59.048640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c3787-f2fc-45f3-bfde-cfaff4121123 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:59.048640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c3787-f2fc-45f3-bfde-cfaff4121123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:59.050207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba7600-1112-4111-8799-6fd7f0181583 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:59.058911 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bace4-c6f7-4b5a-a217-610850936da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:59.060038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bace4-c6f7-4b5a-a217-610850936da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:59.060441 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bace4-c6f7-4b5a-a217-610850936da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:59.061071 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bace4-c6f7-4b5a-a217-610850936da3 admin admin] PATH_INFO: `/v3/projects/ef2d6505ef4545bbbe4df90764eb882e/users/971b38592f2e4384b0ce2e702a33ce45/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:59.091792 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bace4-c6f7-4b5a-a217-610850936da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2d6505ef4545bbbe4df90764eb882e, user_id=971b38592f2e4384b0ce2e702a33ce45, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:59.100303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bace4-c6f7-4b5a-a217-610850936da3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:59.105076 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-8f1c3787-f2fc-45f3-bfde-cfaff4121123 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:59.107976 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8f1c3787-f2fc-45f3-bfde-cfaff4121123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:59.131179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 961/1918] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:59 2022] PUT /identity/v3/projects/ef2d6505ef4545bbbe4df90764eb882e/users/971b38592f2e4384b0ce2e702a33ce45/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:59.133340 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 958/1919] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:55:59.137738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bace4-c6f7-4b5a-a217-610850936da3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:59.146463 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c15d1-320a-499f-8a69-607d14f77794 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:59.147639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c15d1-320a-499f-8a69-607d14f77794 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:59.149516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c15d1-320a-499f-8a69-607d14f77794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:59.244616 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c72c15d1-320a-499f-8a69-607d14f77794 None None] MFA Rules not processed for user `971b38592f2e4384b0ce2e702a33ce45`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:55:59.246009 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c72c15d1-320a-499f-8a69-607d14f77794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:59.286728 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:59.307724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bace4-c6f7-4b5a-a217-610850936da3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:59.319803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d10bc-6f57-4b97-a00d-543c21a489d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:59.321025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d10bc-6f57-4b97-a00d-543c21a489d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:59.321307 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d10bc-6f57-4b97-a00d-543c21a489d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:59.321670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d10bc-6f57-4b97-a00d-543c21a489d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:59.322305 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d10bc-6f57-4b97-a00d-543c21a489d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:55:59.322530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d10bc-6f57-4b97-a00d-543c21a489d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:59.324162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d10bc-6f57-4b97-a00d-543c21a489d2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:59.351805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 962/1920] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:55:59.355296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d10bc-6f57-4b97-a00d-543c21a489d2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:59.377019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d10bc-6f57-4b97-a00d-543c21a489d2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:59.387848 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fb9f3-b72d-4e26-9c73-c45c8fc3e56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:59.388254 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fb9f3-b72d-4e26-9c73-c45c8fc3e56f admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:59.395042 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fb9f3-b72d-4e26-9c73-c45c8fc3e56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:59.395705 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fb9f3-b72d-4e26-9c73-c45c8fc3e56f admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:59.395893 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 959/1921] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:55:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:55:59.396604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fb9f3-b72d-4e26-9c73-c45c8fc3e56f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:59.398718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fb9f3-b72d-4e26-9c73-c45c8fc3e56f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:59.443505 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c15d1-320a-499f-8a69-607d14f77794 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:59.446213 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c72c15d1-320a-499f-8a69-607d14f77794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:59.472646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 963/1922] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:55:59 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:55:59.477383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fb9f3-b72d-4e26-9c73-c45c8fc3e56f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:59.530648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:59.541084 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c15d1-320a-499f-8a69-607d14f77794 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:59.588280 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fb9f3-b72d-4e26-9c73-c45c8fc3e56f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:59.593907 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b26d3-d5b7-4d55-a2ad-57cd926ad175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:59.594949 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b26d3-d5b7-4d55-a2ad-57cd926ad175 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:59.595800 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b26d3-d5b7-4d55-a2ad-57cd926ad175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:59.596155 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b26d3-d5b7-4d55-a2ad-57cd926ad175 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:59.596908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b26d3-d5b7-4d55-a2ad-57cd926ad175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:59.598990 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b26d3-d5b7-4d55-a2ad-57cd926ad175 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:59.622396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6dba2-54d8-4021-b271-810cd4c96efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:59.624365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6dba2-54d8-4021-b271-810cd4c96efe service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:59.624365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6dba2-54d8-4021-b271-810cd4c96efe service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:59.624365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6dba2-54d8-4021-b271-810cd4c96efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:59.628711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 964/1923] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:55:59 2022] POST /identity/v3/users => generated 291 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:55:59.634023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b26d3-d5b7-4d55-a2ad-57cd926ad175 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:59.653314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b26d3-d5b7-4d55-a2ad-57cd926ad175 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:59.661244 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b6cec-ee50-4183-a0b2-7c26693fe7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:59.662059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b6cec-ee50-4183-a0b2-7c26693fe7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:59.662440 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b6cec-ee50-4183-a0b2-7c26693fe7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:59.662807 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b6cec-ee50-4183-a0b2-7c26693fe7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:59.663551 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b6cec-ee50-4183-a0b2-7c26693fe7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:59.666023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b6cec-ee50-4183-a0b2-7c26693fe7b6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:59.686487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 965/1924] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:59.692001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b6cec-ee50-4183-a0b2-7c26693fe7b6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:59.721558 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b6cec-ee50-4183-a0b2-7c26693fe7b6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:59.727878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6dba2-54d8-4021-b271-810cd4c96efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:59.731539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6dba2-54d8-4021-b271-810cd4c96efe service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:59.736660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae56a9-7880-465c-b30c-a5adcbc92fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:59.737423 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae56a9-7880-465c-b30c-a5adcbc92fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:59.737709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae56a9-7880-465c-b30c-a5adcbc92fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:59.738024 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae56a9-7880-465c-b30c-a5adcbc92fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:59.742530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae56a9-7880-465c-b30c-a5adcbc92fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:59.744326 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae56a9-7880-465c-b30c-a5adcbc92fd5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:59.766749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 960/1925] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:55:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 07:55:59.783581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 966/1926] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:55:59.785413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6dba2-54d8-4021-b271-810cd4c96efe service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:59.786299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae56a9-7880-465c-b30c-a5adcbc92fd5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:59.803490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae56a9-7880-465c-b30c-a5adcbc92fd5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:59.817889 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62201c-dbdc-4b62-813d-f2265b7779ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:59.818600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62201c-dbdc-4b62-813d-f2265b7779ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:59.819434 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62201c-dbdc-4b62-813d-f2265b7779ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:59.819434 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62201c-dbdc-4b62-813d-f2265b7779ff admin admin] PATH_INFO: `/v3/users/971b38592f2e4384b0ce2e702a33ce45` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:59.821635 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6dba2-54d8-4021-b271-810cd4c96efe service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:55:59.825082 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62201c-dbdc-4b62-813d-f2265b7779ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=971b38592f2e4384b0ce2e702a33ce45)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:59.827468 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62201c-dbdc-4b62-813d-f2265b7779ff admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:59.829744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f52e2-bf8e-4774-ac93-d573335305c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:55:59.832406 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f52e2-bf8e-4774-ac93-d573335305c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:55:59.832698 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f52e2-bf8e-4774-ac93-d573335305c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:55:59.834111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f52e2-bf8e-4774-ac93-d573335305c9 admin admin] PATH_INFO: `/v3/projects/1d9a526a38394b429f07b6ab4f80ebaa/users/48e839a4bc894e5b92fabc8c6f03c8a9/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:55:59.875285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f52e2-bf8e-4774-ac93-d573335305c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9a526a38394b429f07b6ab4f80ebaa, user_id=48e839a4bc894e5b92fabc8c6f03c8a9, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:55:59.886113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f52e2-bf8e-4774-ac93-d573335305c9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:55:59.928706 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 961/1927] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:55:59 2022] PUT /identity/v3/projects/1d9a526a38394b429f07b6ab4f80ebaa/users/48e839a4bc894e5b92fabc8c6f03c8a9/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:55:59.940203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f52e2-bf8e-4774-ac93-d573335305c9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:55:59.940203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-bc0f52e2-bf8e-4774-ac93-d573335305c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:55:59.946495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:55:59.967792 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7d62201c-dbdc-4b62-813d-f2265b7779ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '971b38592f2e4384b0ce2e702a33ce45'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:59.985630 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7d62201c-dbdc-4b62-813d-f2265b7779ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '971b38592f2e4384b0ce2e702a33ce45'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:59.998139 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7d62201c-dbdc-4b62-813d-f2265b7779ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '971b38592f2e4384b0ce2e702a33ce45'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:55:59.999082 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7d62201c-dbdc-4b62-813d-f2265b7779ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '971b38592f2e4384b0ce2e702a33ce45'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:00.035496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7d62201c-dbdc-4b62-813d-f2265b7779ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '971b38592f2e4384b0ce2e702a33ce45'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:00.035637 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f52e2-bf8e-4774-ac93-d573335305c9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:00.043002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a35c08-d4d7-4786-8e13-eb066698d108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:00.043643 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a35c08-d4d7-4786-8e13-eb066698d108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:00.043919 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a35c08-d4d7-4786-8e13-eb066698d108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:00.044251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 967/1928] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:55:59 2022] DELETE /identity/v3/users/971b38592f2e4384b0ce2e702a33ce45 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:56:00.044360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a35c08-d4d7-4786-8e13-eb066698d108 admin admin] PATH_INFO: `/v3/projects/e8fd78eed2114eb7a2140f609993ff82` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:00.045631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a35c08-d4d7-4786-8e13-eb066698d108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8fd78eed2114eb7a2140f609993ff82)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:00.046196 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728195e-e5a9-4db6-908f-d501fe83b8a6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:00.046558 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728195e-e5a9-4db6-908f-d501fe83b8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:00.046958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728195e-e5a9-4db6-908f-d501fe83b8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:00.047920 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a35c08-d4d7-4786-8e13-eb066698d108 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:00.094422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0728195e-e5a9-4db6-908f-d501fe83b8a6 None None] MFA Rules not processed for user `48e839a4bc894e5b92fabc8c6f03c8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:00.095684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0728195e-e5a9-4db6-908f-d501fe83b8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:00.127350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-93a35c08-d4d7-4786-8e13-eb066698d108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8fd78eed2114eb7a2140f609993ff82', 'request_id': 'req-93a35c08-d4d7-4786-8e13-eb066698d108'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:00.133824 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-93a35c08-d4d7-4786-8e13-eb066698d108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8fd78eed2114eb7a2140f609993ff82', 'request_id': 'req-93a35c08-d4d7-4786-8e13-eb066698d108'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:00.141598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-93a35c08-d4d7-4786-8e13-eb066698d108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8fd78eed2114eb7a2140f609993ff82', 'request_id': 'req-93a35c08-d4d7-4786-8e13-eb066698d108'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:00.155262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:00.155587 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-93a35c08-d4d7-4786-8e13-eb066698d108 admin admin] The token cache is being invalidate because project e8fd78eed2114eb7a2140f609993ff82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:00.155791 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-93a35c08-d4d7-4786-8e13-eb066698d108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:00.156569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-93a35c08-d4d7-4786-8e13-eb066698d108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:00.161016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 962/1929] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:55:59 2022] DELETE /identity/v3/projects/e8fd78eed2114eb7a2140f609993ff82 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:00.161016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8711794-f171-4cd0-a506-2a6fca0680f3 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:00.161016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8711794-f171-4cd0-a506-2a6fca0680f3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:00.161016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8711794-f171-4cd0-a506-2a6fca0680f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:00.180599 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c8711794-f171-4cd0-a506-2a6fca0680f3 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:00.181823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c8711794-f171-4cd0-a506-2a6fca0680f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:00.182959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 968/1930] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:00.184314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728195e-e5a9-4db6-908f-d501fe83b8a6 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:00.185851 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0728195e-e5a9-4db6-908f-d501fe83b8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:00.271200 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728195e-e5a9-4db6-908f-d501fe83b8a6 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:00.293037 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 963/1931] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:00.294604 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a043c9-4ca1-4744-a54f-96202f0017a3 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:00.295436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a043c9-4ca1-4744-a54f-96202f0017a3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:00.295972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a043c9-4ca1-4744-a54f-96202f0017a3 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:00.297464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 964/1932] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 7 07:56:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:56:00.298250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a043c9-4ca1-4744-a54f-96202f0017a3 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:00.300956 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e8a043c9-4ca1-4744-a54f-96202f0017a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:00.319374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a043c9-4ca1-4744-a54f-96202f0017a3 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:00.324938 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a58ca7-0ee6-4e4b-834a-5aa12063cac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:00.325684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a58ca7-0ee6-4e4b-834a-5aa12063cac0 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:00.326063 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a58ca7-0ee6-4e4b-834a-5aa12063cac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:00.326445 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a58ca7-0ee6-4e4b-834a-5aa12063cac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:00.327184 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a58ca7-0ee6-4e4b-834a-5aa12063cac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:00.329579 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a58ca7-0ee6-4e4b-834a-5aa12063cac0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:00.367051 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18e88b-6766-4132-9ee1-88b40d6c8021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:00.367872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18e88b-6766-4132-9ee1-88b40d6c8021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:00.368194 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18e88b-6766-4132-9ee1-88b40d6c8021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:00.368508 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18e88b-6766-4132-9ee1-88b40d6c8021 admin admin] PATH_INFO: `/v3/users/cafecd37bb104cba934d51608a0178cc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:00.369182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 965/1933] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:00 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:00.370386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18e88b-6766-4132-9ee1-88b40d6c8021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cafecd37bb104cba934d51608a0178cc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:00.370522 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a58ca7-0ee6-4e4b-834a-5aa12063cac0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:00.372416 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18e88b-6766-4132-9ee1-88b40d6c8021 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:00.373364 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-72a58ca7-0ee6-4e4b-834a-5aa12063cac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:00.396856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:00.467916 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6d18e88b-6766-4132-9ee1-88b40d6c8021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cafecd37bb104cba934d51608a0178cc'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:00.482403 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a58ca7-0ee6-4e4b-834a-5aa12063cac0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:00.485979 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6d18e88b-6766-4132-9ee1-88b40d6c8021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cafecd37bb104cba934d51608a0178cc'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:00.487081 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa2662-605c-449d-bf63-cc1582d2012c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:00.487971 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa2662-605c-449d-bf63-cc1582d2012c service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:00.488806 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa2662-605c-449d-bf63-cc1582d2012c service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:00.489152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa2662-605c-449d-bf63-cc1582d2012c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:00.499597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6d18e88b-6766-4132-9ee1-88b40d6c8021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cafecd37bb104cba934d51608a0178cc'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:00.500643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6d18e88b-6766-4132-9ee1-88b40d6c8021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cafecd37bb104cba934d51608a0178cc'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:00.520732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6d18e88b-6766-4132-9ee1-88b40d6c8021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cafecd37bb104cba934d51608a0178cc'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:00.530369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 969/1934] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:00 2022] DELETE /identity/v3/users/cafecd37bb104cba934d51608a0178cc => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:00.530369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e6495-544b-4adf-b77e-2a3bb73ee3ee None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:00.530369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e6495-544b-4adf-b77e-2a3bb73ee3ee None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:00.530369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e6495-544b-4adf-b77e-2a3bb73ee3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:00.555841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0b0e6495-544b-4adf-b77e-2a3bb73ee3ee None None] MFA Rules not processed for user `81d5689e9a344bafb74ca423ffd2b42a`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:00.557434 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0b0e6495-544b-4adf-b77e-2a3bb73ee3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:00.586918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:00.639291 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa2662-605c-449d-bf63-cc1582d2012c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:00.640785 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa2662-605c-449d-bf63-cc1582d2012c service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:00.663115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 966/1935] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:00.664256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa2662-605c-449d-bf63-cc1582d2012c service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:00.674960 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa2662-605c-449d-bf63-cc1582d2012c service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:00.702752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 970/1936] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 7 07:56:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:00.717146 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e6495-544b-4adf-b77e-2a3bb73ee3ee None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:00.721107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0b0e6495-544b-4adf-b77e-2a3bb73ee3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:00.741680 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e6495-544b-4adf-b77e-2a3bb73ee3ee None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:00.749213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03b85b-52c1-46a4-be8a-fff4cf781645 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:00.750178 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03b85b-52c1-46a4-be8a-fff4cf781645 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:00.750608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03b85b-52c1-46a4-be8a-fff4cf781645 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:00.750885 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03b85b-52c1-46a4-be8a-fff4cf781645 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:00.763043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03b85b-52c1-46a4-be8a-fff4cf781645 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:00.764277 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03b85b-52c1-46a4-be8a-fff4cf781645 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:00.765654 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dc790-4b00-484f-bf9e-4ca8e4de3d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:00.768344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dc790-4b00-484f-bf9e-4ca8e4de3d46 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:00.770255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dc790-4b00-484f-bf9e-4ca8e4de3d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:00.770255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dc790-4b00-484f-bf9e-4ca8e4de3d46 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:00.771053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140dc790-4b00-484f-bf9e-4ca8e4de3d46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:00.777159 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140dc790-4b00-484f-bf9e-4ca8e4de3d46 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:00.788338 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 971/1937] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:00.817252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 967/1938] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:00 2022] POST /identity/v3/users => generated 298 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:00.822150 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03b85b-52c1-46a4-be8a-fff4cf781645 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:00.823795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3d03b85b-52c1-46a4-be8a-fff4cf781645 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:00.850974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03b85b-52c1-46a4-be8a-fff4cf781645 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:00.865913 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e573e2-a42f-4438-b66f-44b8b6cba580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:00.866530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e573e2-a42f-4438-b66f-44b8b6cba580 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:00.869563 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e573e2-a42f-4438-b66f-44b8b6cba580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:00.869860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e573e2-a42f-4438-b66f-44b8b6cba580 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:00.881957 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e573e2-a42f-4438-b66f-44b8b6cba580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:00.884728 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e573e2-a42f-4438-b66f-44b8b6cba580 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:00.907267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 972/1939] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:00.910656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dc790-4b00-484f-bf9e-4ca8e4de3d46 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:00.954736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dc790-4b00-484f-bf9e-4ca8e4de3d46 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:00.975475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85405d4a-3661-454e-8f3d-52da858f4f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:00.976683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85405d4a-3661-454e-8f3d-52da858f4f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:00.976988 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85405d4a-3661-454e-8f3d-52da858f4f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:00.977248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85405d4a-3661-454e-8f3d-52da858f4f83 admin admin] PATH_INFO: `/v3/projects/f6f89f9594ab444e8c4b5847b9ce5473/users/ee955e16f37945cc92c123f1ff3aef45/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:01.020714 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85405d4a-3661-454e-8f3d-52da858f4f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6f89f9594ab444e8c4b5847b9ce5473, user_id=ee955e16f37945cc92c123f1ff3aef45, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:01.024916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85405d4a-3661-454e-8f3d-52da858f4f83 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:01.074695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 968/1940] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:00 2022] PUT /identity/v3/projects/f6f89f9594ab444e8c4b5847b9ce5473/users/ee955e16f37945cc92c123f1ff3aef45/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:01.078471 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e573e2-a42f-4438-b66f-44b8b6cba580 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:01.096653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e573e2-a42f-4438-b66f-44b8b6cba580 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:01.186734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15551320-8693-4103-9fcc-f53ea9d1069a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:01.187763 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15551320-8693-4103-9fcc-f53ea9d1069a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:01.187898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15551320-8693-4103-9fcc-f53ea9d1069a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:01.188336 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15551320-8693-4103-9fcc-f53ea9d1069a admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:01.188991 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15551320-8693-4103-9fcc-f53ea9d1069a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:01.192613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15551320-8693-4103-9fcc-f53ea9d1069a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:01.216432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85405d4a-3661-454e-8f3d-52da858f4f83 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:01.216866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 973/1941] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:01.229007 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15551320-8693-4103-9fcc-f53ea9d1069a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:01.230688 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85405d4a-3661-454e-8f3d-52da858f4f83 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:01.231459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-15551320-8693-4103-9fcc-f53ea9d1069a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:01.237497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015bd845-4e43-4532-b97c-6fddb88152b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:01.238437 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bd845-4e43-4532-b97c-6fddb88152b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:01.239534 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bd845-4e43-4532-b97c-6fddb88152b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:01.239912 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bd845-4e43-4532-b97c-6fddb88152b5 admin admin] PATH_INFO: `/v3/projects/f6f89f9594ab444e8c4b5847b9ce5473/users/ee955e16f37945cc92c123f1ff3aef45/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:01.244118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015bd845-4e43-4532-b97c-6fddb88152b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6f89f9594ab444e8c4b5847b9ce5473, user_id=ee955e16f37945cc92c123f1ff3aef45, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:01.249692 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015bd845-4e43-4532-b97c-6fddb88152b5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:01.255076 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15551320-8693-4103-9fcc-f53ea9d1069a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:01.262615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1ec6c-79ee-4340-8557-69860e610452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:01.263868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1ec6c-79ee-4340-8557-69860e610452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:01.264396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1ec6c-79ee-4340-8557-69860e610452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:01.265242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1ec6c-79ee-4340-8557-69860e610452 admin admin] PATH_INFO: `/v3/projects/ef2d6505ef4545bbbe4df90764eb882e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:01.267922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1ec6c-79ee-4340-8557-69860e610452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2d6505ef4545bbbe4df90764eb882e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:01.270698 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1ec6c-79ee-4340-8557-69860e610452 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:01.294071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 969/1942] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:01 2022] PUT /identity/v3/projects/f6f89f9594ab444e8c4b5847b9ce5473/users/ee955e16f37945cc92c123f1ff3aef45/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:56:01.295778 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c21c8-bbfc-471a-8b35-ab0f0527e6bd None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:01.296204 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c21c8-bbfc-471a-8b35-ab0f0527e6bd None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:01.297345 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c21c8-bbfc-471a-8b35-ab0f0527e6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:01.346432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-98c1ec6c-79ee-4340-8557-69860e610452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2d6505ef4545bbbe4df90764eb882e', 'request_id': 'req-98c1ec6c-79ee-4340-8557-69860e610452'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:01.353079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-98c1ec6c-79ee-4340-8557-69860e610452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2d6505ef4545bbbe4df90764eb882e', 'request_id': 'req-98c1ec6c-79ee-4340-8557-69860e610452'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:01.360391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-98c1ec6c-79ee-4340-8557-69860e610452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2d6505ef4545bbbe4df90764eb882e', 'request_id': 'req-98c1ec6c-79ee-4340-8557-69860e610452'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:01.368660 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-dd8c21c8-bbfc-471a-8b35-ab0f0527e6bd None None] MFA Rules not processed for user `ee955e16f37945cc92c123f1ff3aef45`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:01.369987 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dd8c21c8-bbfc-471a-8b35-ab0f0527e6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:01.374541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:01.383604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-98c1ec6c-79ee-4340-8557-69860e610452 admin admin] The token cache is being invalidate because project ef2d6505ef4545bbbe4df90764eb882e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:01.383604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-98c1ec6c-79ee-4340-8557-69860e610452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:01.383604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-98c1ec6c-79ee-4340-8557-69860e610452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:01.389126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 974/1943] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:01 2022] DELETE /identity/v3/projects/ef2d6505ef4545bbbe4df90764eb882e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:01.392997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f6c03-d577-4954-80a7-ec01c591f572 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:01.393430 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f6c03-d577-4954-80a7-ec01c591f572 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:01.393884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f6c03-d577-4954-80a7-ec01c591f572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:01.404858 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:01.424807 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a05f6c03-d577-4954-80a7-ec01c591f572 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:01.426035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a05f6c03-d577-4954-80a7-ec01c591f572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:01.471542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 970/1944] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:01 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:01.472825 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c21c8-bbfc-471a-8b35-ab0f0527e6bd None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:01.475769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dd8c21c8-bbfc-471a-8b35-ab0f0527e6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:01.502445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 975/1945] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:01.503432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f6c03-d577-4954-80a7-ec01c591f572 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:01.504800 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a05f6c03-d577-4954-80a7-ec01c591f572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:01.555336 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c21c8-bbfc-471a-8b35-ab0f0527e6bd None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:01.565210 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d87e1-6030-421d-bd49-0100472538b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:01.565377 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f6c03-d577-4954-80a7-ec01c591f572 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:01.565887 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d87e1-6030-421d-bd49-0100472538b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:01.566168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d87e1-6030-421d-bd49-0100472538b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:01.566893 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d87e1-6030-421d-bd49-0100472538b5 admin admin] PATH_INFO: `/v3/projects/3d9edb804c804c07a89f17c302a6b8b3` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:01.568205 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d87e1-6030-421d-bd49-0100472538b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d9edb804c804c07a89f17c302a6b8b3)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:01.569283 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad606d-86ed-4cd3-94c8-138c26913dce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:01.569918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad606d-86ed-4cd3-94c8-138c26913dce service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:01.570129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d87e1-6030-421d-bd49-0100472538b5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:01.570235 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad606d-86ed-4cd3-94c8-138c26913dce service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:01.570455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad606d-86ed-4cd3-94c8-138c26913dce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:01.587346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad606d-86ed-4cd3-94c8-138c26913dce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:01.588922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad606d-86ed-4cd3-94c8-138c26913dce service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:01.599555 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 976/1946] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:01.602216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7dd345-6dea-402b-8c2f-dc7088dcc584 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:01.602405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7dd345-6dea-402b-8c2f-dc7088dcc584 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:01.602828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7dd345-6dea-402b-8c2f-dc7088dcc584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:01.634619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6e7dd345-6dea-402b-8c2f-dc7088dcc584 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:01.634619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6e7dd345-6dea-402b-8c2f-dc7088dcc584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:01.644047 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 977/1947] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:01.647615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7dd345-6dea-402b-8c2f-dc7088dcc584 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:01.664600 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-922d87e1-6030-421d-bd49-0100472538b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d9edb804c804c07a89f17c302a6b8b3', 'request_id': 'req-922d87e1-6030-421d-bd49-0100472538b5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:01.671544 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-922d87e1-6030-421d-bd49-0100472538b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d9edb804c804c07a89f17c302a6b8b3', 'request_id': 'req-922d87e1-6030-421d-bd49-0100472538b5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:01.679772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-922d87e1-6030-421d-bd49-0100472538b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d9edb804c804c07a89f17c302a6b8b3', 'request_id': 'req-922d87e1-6030-421d-bd49-0100472538b5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:01.687321 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7dd345-6dea-402b-8c2f-dc7088dcc584 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:01.700328 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-922d87e1-6030-421d-bd49-0100472538b5 admin admin] The token cache is being invalidate because project 3d9edb804c804c07a89f17c302a6b8b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:01.700680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-922d87e1-6030-421d-bd49-0100472538b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:01.701438 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-922d87e1-6030-421d-bd49-0100472538b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:01.703340 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 971/1948] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:01 2022] DELETE /identity/v3/projects/3d9edb804c804c07a89f17c302a6b8b3 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:01.709226 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef8fd7-6fa2-4001-9402-40b95705b3c0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:01.712124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef8fd7-6fa2-4001-9402-40b95705b3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:01.712124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef8fd7-6fa2-4001-9402-40b95705b3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:01.735003 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-01ef8fd7-6fa2-4001-9402-40b95705b3c0 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:01.736353 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-01ef8fd7-6fa2-4001-9402-40b95705b3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:01.768841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:01.780466 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848ac73-6683-47c2-a42b-0f735c7c93dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:01.781084 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848ac73-6683-47c2-a42b-0f735c7c93dc admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:01.781364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848ac73-6683-47c2-a42b-0f735c7c93dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:01.781635 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848ac73-6683-47c2-a42b-0f735c7c93dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:01.782281 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848ac73-6683-47c2-a42b-0f735c7c93dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:56:01.782589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848ac73-6683-47c2-a42b-0f735c7c93dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:01.784192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848ac73-6683-47c2-a42b-0f735c7c93dc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:01.805411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 978/1949] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:56:01.806434 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848ac73-6683-47c2-a42b-0f735c7c93dc admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:01.808276 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7848ac73-6683-47c2-a42b-0f735c7c93dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:01.812717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 972/1950] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:56:01.813534 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef8fd7-6fa2-4001-9402-40b95705b3c0 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:01.814773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-01ef8fd7-6fa2-4001-9402-40b95705b3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:01.827170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef8fd7-6fa2-4001-9402-40b95705b3c0 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:01.831749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208e89f-f0d0-42ce-aa3d-8c30fc25702e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:01.832395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208e89f-f0d0-42ce-aa3d-8c30fc25702e admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:01.832732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208e89f-f0d0-42ce-aa3d-8c30fc25702e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:01.833011 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208e89f-f0d0-42ce-aa3d-8c30fc25702e admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:01.833634 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208e89f-f0d0-42ce-aa3d-8c30fc25702e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:01.835790 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208e89f-f0d0-42ce-aa3d-8c30fc25702e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:01.850525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:01.863065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 973/1951] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:01 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:01.865761 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04f1e1-dc6f-4a09-9ecf-b41e73f80257 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:01.866340 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04f1e1-dc6f-4a09-9ecf-b41e73f80257 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:01.867269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04f1e1-dc6f-4a09-9ecf-b41e73f80257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:01.875656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848ac73-6683-47c2-a42b-0f735c7c93dc admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:01.894314 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-0d04f1e1-dc6f-4a09-9ecf-b41e73f80257 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:01.894914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0d04f1e1-dc6f-4a09-9ecf-b41e73f80257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:01.926594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade34f7-3f2b-49a9-a498-bbeb17ecedea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:01.927303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade34f7-3f2b-49a9-a498-bbeb17ecedea service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:01.927646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade34f7-3f2b-49a9-a498-bbeb17ecedea service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:01.928498 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade34f7-3f2b-49a9-a498-bbeb17ecedea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:01.931591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-dade34f7-3f2b-49a9-a498-bbeb17ecedea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:01.963475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 974/1952] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:01.964977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04f1e1-dc6f-4a09-9ecf-b41e73f80257 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:01.975810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04f1e1-dc6f-4a09-9ecf-b41e73f80257 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:01.988410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ce0d6-512c-424f-b71e-2901a9754dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:01.989515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ce0d6-512c-424f-b71e-2901a9754dce admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:01.989847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ce0d6-512c-424f-b71e-2901a9754dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:01.990130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ce0d6-512c-424f-b71e-2901a9754dce admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:01.991402 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ce0d6-512c-424f-b71e-2901a9754dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:01.994902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ce0d6-512c-424f-b71e-2901a9754dce admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:02.009984 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade34f7-3f2b-49a9-a498-bbeb17ecedea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:02.012306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade34f7-3f2b-49a9-a498-bbeb17ecedea service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:02.027698 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 975/1953] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:01 2022] POST /identity/v3/users => generated 288 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:02.027698 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ce0d6-512c-424f-b71e-2901a9754dce admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:02.027973 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 979/1954] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:02.028870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade34f7-3f2b-49a9-a498-bbeb17ecedea service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:02.041245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade34f7-3f2b-49a9-a498-bbeb17ecedea service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:02.044841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ce0d6-512c-424f-b71e-2901a9754dce admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:02.046270 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663f645-196b-4c62-9ee1-b6d16fd87748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:02.046878 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663f645-196b-4c62-9ee1-b6d16fd87748 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:02.047152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663f645-196b-4c62-9ee1-b6d16fd87748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:02.047558 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663f645-196b-4c62-9ee1-b6d16fd87748 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:02.048236 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663f645-196b-4c62-9ee1-b6d16fd87748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:02.049805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663f645-196b-4c62-9ee1-b6d16fd87748 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:02.049927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517891e-c959-4520-8e1c-ec26002fca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:02.050545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517891e-c959-4520-8e1c-ec26002fca87 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:02.050821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517891e-c959-4520-8e1c-ec26002fca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:02.051095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517891e-c959-4520-8e1c-ec26002fca87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:02.051800 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517891e-c959-4520-8e1c-ec26002fca87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:56:02.052136 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517891e-c959-4520-8e1c-ec26002fca87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:02.053867 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517891e-c959-4520-8e1c-ec26002fca87 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:02.061271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 980/1955] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:02.064996 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663f645-196b-4c62-9ee1-b6d16fd87748 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:02.071270 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 976/1956] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:56:02.075445 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517891e-c959-4520-8e1c-ec26002fca87 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:02.075999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663f645-196b-4c62-9ee1-b6d16fd87748 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:02.081001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fe1e4-1b8c-4555-909e-bda90489d6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:02.081674 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fe1e4-1b8c-4555-909e-bda90489d6b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:02.082022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fe1e4-1b8c-4555-909e-bda90489d6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:02.082362 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fe1e4-1b8c-4555-909e-bda90489d6b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:02.083070 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53fe1e4-1b8c-4555-909e-bda90489d6b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:02.084752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53fe1e4-1b8c-4555-909e-bda90489d6b6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:02.092295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517891e-c959-4520-8e1c-ec26002fca87 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:02.096544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 981/1957] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:56:02.098431 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbedc9-1952-4c29-af2f-8d2df5baf705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:02.099110 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbedc9-1952-4c29-af2f-8d2df5baf705 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:02.099386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbedc9-1952-4c29-af2f-8d2df5baf705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:02.099716 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbedc9-1952-4c29-af2f-8d2df5baf705 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:02.100397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbedc9-1952-4c29-af2f-8d2df5baf705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:02.102626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fe1e4-1b8c-4555-909e-bda90489d6b6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:02.103087 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbedc9-1952-4c29-af2f-8d2df5baf705 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:02.115047 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fe1e4-1b8c-4555-909e-bda90489d6b6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:02.122260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1382c66-53e4-480f-95ae-b03eda74bcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:02.123041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1382c66-53e4-480f-95ae-b03eda74bcfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:02.123321 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1382c66-53e4-480f-95ae-b03eda74bcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:02.123600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1382c66-53e4-480f-95ae-b03eda74bcfb admin admin] PATH_INFO: `/v3/projects/8e646215bae9482d9b1e00b35dd8b20e/users/e72ae4e2fa4541aa8b046ddba2bc0a3b/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:02.132371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 977/1958] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:02 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:56:02.136448 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbedc9-1952-4c29-af2f-8d2df5baf705 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:02.142941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1382c66-53e4-480f-95ae-b03eda74bcfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e646215bae9482d9b1e00b35dd8b20e, user_id=e72ae4e2fa4541aa8b046ddba2bc0a3b, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:02.145713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1382c66-53e4-480f-95ae-b03eda74bcfb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:02.161361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 982/1959] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:02 2022] PUT /identity/v3/projects/8e646215bae9482d9b1e00b35dd8b20e/users/e72ae4e2fa4541aa8b046ddba2bc0a3b/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:02.172168 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecd543-e5e8-464f-af51-05372c5f1ade None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:02.172460 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecd543-e5e8-464f-af51-05372c5f1ade None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:02.172759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecd543-e5e8-464f-af51-05372c5f1ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:02.203700 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a8ecd543-e5e8-464f-af51-05372c5f1ade None None] MFA Rules not processed for user `e72ae4e2fa4541aa8b046ddba2bc0a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:02.204961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a8ecd543-e5e8-464f-af51-05372c5f1ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:02.214754 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:02.219492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbedc9-1952-4c29-af2f-8d2df5baf705 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:02.255536 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:02.280195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 983/1960] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:56:02.292886 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecd543-e5e8-464f-af51-05372c5f1ade None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:02.294528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a8ecd543-e5e8-464f-af51-05372c5f1ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:02.295334 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32859372-e3ba-4eec-bf74-4121a8ca22ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:02.295946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32859372-e3ba-4eec-bf74-4121a8ca22ee admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:02.296271 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32859372-e3ba-4eec-bf74-4121a8ca22ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:02.296541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32859372-e3ba-4eec-bf74-4121a8ca22ee admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:02.297161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32859372-e3ba-4eec-bf74-4121a8ca22ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:02.299097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32859372-e3ba-4eec-bf74-4121a8ca22ee admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:02.330160 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 978/1961] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:02 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:56:02.331556 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32859372-e3ba-4eec-bf74-4121a8ca22ee admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:02.345677 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32859372-e3ba-4eec-bf74-4121a8ca22ee admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:02.374220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cab04-4c78-4e06-9bc0-f96a86bc7e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:02.374846 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cab04-4c78-4e06-9bc0-f96a86bc7e24 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:02.375120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cab04-4c78-4e06-9bc0-f96a86bc7e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:02.375877 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cab04-4c78-4e06-9bc0-f96a86bc7e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:02.376465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8cab04-4c78-4e06-9bc0-f96a86bc7e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:02.381338 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8cab04-4c78-4e06-9bc0-f96a86bc7e24 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:02.385519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecd543-e5e8-464f-af51-05372c5f1ade None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:02.390748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63201f63-5dc4-49d5-bd01-eacc5984e37c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:02.391495 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63201f63-5dc4-49d5-bd01-eacc5984e37c service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:02.391875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63201f63-5dc4-49d5-bd01-eacc5984e37c service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:02.392443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63201f63-5dc4-49d5-bd01-eacc5984e37c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:02.400522 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 979/1962] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:02.401662 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cab04-4c78-4e06-9bc0-f96a86bc7e24 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:02.403636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63201f63-5dc4-49d5-bd01-eacc5984e37c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:02.406548 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63201f63-5dc4-49d5-bd01-eacc5984e37c service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:02.425026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cab04-4c78-4e06-9bc0-f96a86bc7e24 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:02.426186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 984/1963] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 07:56:02.432296 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764431b2-2272-49bc-9d63-e3a01cd2f4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:02.433085 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764431b2-2272-49bc-9d63-e3a01cd2f4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:02.433285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764431b2-2272-49bc-9d63-e3a01cd2f4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:02.433561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764431b2-2272-49bc-9d63-e3a01cd2f4c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:02.434336 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764431b2-2272-49bc-9d63-e3a01cd2f4c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:02.435992 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764431b2-2272-49bc-9d63-e3a01cd2f4c1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:02.456758 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63201f63-5dc4-49d5-bd01-eacc5984e37c service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:02.463042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 980/1964] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:02.469155 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63201f63-5dc4-49d5-bd01-eacc5984e37c service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:02.479554 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a65a2-e711-4475-a388-3bb6dacdefb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:02.481187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a65a2-e711-4475-a388-3bb6dacdefb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:02.481686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a65a2-e711-4475-a388-3bb6dacdefb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:02.482041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a65a2-e711-4475-a388-3bb6dacdefb4 admin admin] PATH_INFO: `/v3/projects/42bfe62281da4d28b580ef2caadade6f/users/ca084462cad946c5909274fd33d33238/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:02.501373 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a65a2-e711-4475-a388-3bb6dacdefb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42bfe62281da4d28b580ef2caadade6f, user_id=ca084462cad946c5909274fd33d33238, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:02.504556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a65a2-e711-4475-a388-3bb6dacdefb4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:02.526332 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 985/1965] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:02 2022] PUT /identity/v3/projects/42bfe62281da4d28b580ef2caadade6f/users/ca084462cad946c5909274fd33d33238/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:02.536743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc9fc2-16bb-44f8-b73e-4d3aa6dc50da None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:02.537507 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc9fc2-16bb-44f8-b73e-4d3aa6dc50da None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:02.538578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc9fc2-16bb-44f8-b73e-4d3aa6dc50da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:02.583646 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-49fc9fc2-16bb-44f8-b73e-4d3aa6dc50da None None] MFA Rules not processed for user `ca084462cad946c5909274fd33d33238`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:02.584958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-49fc9fc2-16bb-44f8-b73e-4d3aa6dc50da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:02.660879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 981/1966] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:02.672895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a65a2-e711-4475-a388-3bb6dacdefb4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:02.735909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a65a2-e711-4475-a388-3bb6dacdefb4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:02.740687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f52d17-9cac-4c2a-bc3c-d3494616cd28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:02.741504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f52d17-9cac-4c2a-bc3c-d3494616cd28 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:02.741935 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f52d17-9cac-4c2a-bc3c-d3494616cd28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:02.742210 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f52d17-9cac-4c2a-bc3c-d3494616cd28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:02.756274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f52d17-9cac-4c2a-bc3c-d3494616cd28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:02.756274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f52d17-9cac-4c2a-bc3c-d3494616cd28 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:02.764694 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 986/1967] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:02.788192 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc9fc2-16bb-44f8-b73e-4d3aa6dc50da None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:02.885257 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc9fc2-16bb-44f8-b73e-4d3aa6dc50da None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:02.890614 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abae5a-680f-4dd8-86ab-7721d6b56394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:02.891365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abae5a-680f-4dd8-86ab-7721d6b56394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:02.891684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abae5a-680f-4dd8-86ab-7721d6b56394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:02.891964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abae5a-680f-4dd8-86ab-7721d6b56394 admin admin] PATH_INFO: `/v3/users/ca084462cad946c5909274fd33d33238` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:02.893180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abae5a-680f-4dd8-86ab-7721d6b56394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca084462cad946c5909274fd33d33238)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:02.895097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abae5a-680f-4dd8-86ab-7721d6b56394 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:02.899177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:02.954674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-10abae5a-680f-4dd8-86ab-7721d6b56394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca084462cad946c5909274fd33d33238'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:02.955360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-10abae5a-680f-4dd8-86ab-7721d6b56394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca084462cad946c5909274fd33d33238'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:02.968032 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-10abae5a-680f-4dd8-86ab-7721d6b56394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca084462cad946c5909274fd33d33238'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:02.968822 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-10abae5a-680f-4dd8-86ab-7721d6b56394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca084462cad946c5909274fd33d33238'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:02.975543 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-10abae5a-680f-4dd8-86ab-7721d6b56394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca084462cad946c5909274fd33d33238'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:02.989056 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 982/1968] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:02 2022] DELETE /identity/v3/users/ca084462cad946c5909274fd33d33238 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:02.994631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd43f0-6906-4796-8564-3fade9132629 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:02.994915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd43f0-6906-4796-8564-3fade9132629 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:02.995236 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd43f0-6906-4796-8564-3fade9132629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:03.015071 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7ffd43f0-6906-4796-8564-3fade9132629 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:03.016497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7ffd43f0-6906-4796-8564-3fade9132629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:03.075564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 987/1969] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:03.085254 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abae5a-680f-4dd8-86ab-7721d6b56394 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:03.087265 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-10abae5a-680f-4dd8-86ab-7721d6b56394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:03.142706 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abae5a-680f-4dd8-86ab-7721d6b56394 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:03.146314 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cb3fa-bb04-4e51-a86c-c44a44453257 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:03.146916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cb3fa-bb04-4e51-a86c-c44a44453257 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:03.147195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cb3fa-bb04-4e51-a86c-c44a44453257 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:03.147468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cb3fa-bb04-4e51-a86c-c44a44453257 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:03.156876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cb3fa-bb04-4e51-a86c-c44a44453257 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:03.158306 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cb3fa-bb04-4e51-a86c-c44a44453257 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:03.168213 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 983/1970] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:03.449287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd43f0-6906-4796-8564-3fade9132629 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:03.451327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7ffd43f0-6906-4796-8564-3fade9132629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:03.468007 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd43f0-6906-4796-8564-3fade9132629 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:03.472813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0a5bc-a552-4dee-b335-30ce5ce42489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:03.473460 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0a5bc-a552-4dee-b335-30ce5ce42489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:03.473741 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0a5bc-a552-4dee-b335-30ce5ce42489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:03.474015 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0a5bc-a552-4dee-b335-30ce5ce42489 admin admin] PATH_INFO: `/v3/projects/42bfe62281da4d28b580ef2caadade6f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:03.475185 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0a5bc-a552-4dee-b335-30ce5ce42489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42bfe62281da4d28b580ef2caadade6f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:03.477166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0a5bc-a552-4dee-b335-30ce5ce42489 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:03.479616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:03.524613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-46d0a5bc-a552-4dee-b335-30ce5ce42489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42bfe62281da4d28b580ef2caadade6f', 'request_id': 'req-46d0a5bc-a552-4dee-b335-30ce5ce42489'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:03.530089 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-46d0a5bc-a552-4dee-b335-30ce5ce42489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42bfe62281da4d28b580ef2caadade6f', 'request_id': 'req-46d0a5bc-a552-4dee-b335-30ce5ce42489'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:03.536068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-46d0a5bc-a552-4dee-b335-30ce5ce42489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42bfe62281da4d28b580ef2caadade6f', 'request_id': 'req-46d0a5bc-a552-4dee-b335-30ce5ce42489'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:03.547909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-46d0a5bc-a552-4dee-b335-30ce5ce42489 admin admin] The token cache is being invalidate because project 42bfe62281da4d28b580ef2caadade6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:03.548239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-46d0a5bc-a552-4dee-b335-30ce5ce42489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:03.548877 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-46d0a5bc-a552-4dee-b335-30ce5ce42489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:03.550345 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 988/1971] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:03 2022] DELETE /identity/v3/projects/42bfe62281da4d28b580ef2caadade6f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:03.555547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84b9b7-ccc4-4955-8273-92f95f4fe825 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:03.555834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84b9b7-ccc4-4955-8273-92f95f4fe825 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:03.556163 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84b9b7-ccc4-4955-8273-92f95f4fe825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:03.577056 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1b84b9b7-ccc4-4955-8273-92f95f4fe825 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:03.578072 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1b84b9b7-ccc4-4955-8273-92f95f4fe825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:03.639261 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 984/1972] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:03.647926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31a91c-f5cb-4554-aade-a85a5d1ce07d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:03.648762 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31a91c-f5cb-4554-aade-a85a5d1ce07d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:03.649034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31a91c-f5cb-4554-aade-a85a5d1ce07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:03.670829 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8e31a91c-f5cb-4554-aade-a85a5d1ce07d None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:03.672055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8e31a91c-f5cb-4554-aade-a85a5d1ce07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:03.682210 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 989/1973] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:03.686887 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84b9b7-ccc4-4955-8273-92f95f4fe825 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:03.697285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84b9b7-ccc4-4955-8273-92f95f4fe825 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:03.702833 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32016b1-49e2-49c5-9178-553b6553876c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:03.703509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32016b1-49e2-49c5-9178-553b6553876c admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:03.703722 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32016b1-49e2-49c5-9178-553b6553876c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:03.703994 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32016b1-49e2-49c5-9178-553b6553876c admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:03.704674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32016b1-49e2-49c5-9178-553b6553876c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:56:03.704974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32016b1-49e2-49c5-9178-553b6553876c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:03.706581 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32016b1-49e2-49c5-9178-553b6553876c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:03.724120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 985/1974] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:56:03.726988 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31a91c-f5cb-4554-aade-a85a5d1ce07d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:03.744576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31a91c-f5cb-4554-aade-a85a5d1ce07d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:03.750383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721af5a-08b7-4c74-8696-f69a35431ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:03.751868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721af5a-08b7-4c74-8696-f69a35431ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:03.752469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721af5a-08b7-4c74-8696-f69a35431ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:03.753155 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721af5a-08b7-4c74-8696-f69a35431ab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:03.754023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721af5a-08b7-4c74-8696-f69a35431ab4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:03.756648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721af5a-08b7-4c74-8696-f69a35431ab4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:03.793696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 990/1975] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:03 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:03.797037 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32016b1-49e2-49c5-9178-553b6553876c admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:03.808906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32016b1-49e2-49c5-9178-553b6553876c admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:03.820112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:03.910530 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a8710-d1b4-43db-b9ff-2970c1ce3845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:03.911145 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a8710-d1b4-43db-b9ff-2970c1ce3845 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:03.911426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a8710-d1b4-43db-b9ff-2970c1ce3845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:03.911704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a8710-d1b4-43db-b9ff-2970c1ce3845 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:03.912382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a8710-d1b4-43db-b9ff-2970c1ce3845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:03.914944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a8710-d1b4-43db-b9ff-2970c1ce3845 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:03.946871 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 986/1976] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:03 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:56:03.948039 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721af5a-08b7-4c74-8696-f69a35431ab4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:03.967395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721af5a-08b7-4c74-8696-f69a35431ab4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:03.995866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b619a-8052-4aa3-a8b8-20060464a118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:03.997223 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b619a-8052-4aa3-a8b8-20060464a118 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:03.999343 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b619a-8052-4aa3-a8b8-20060464a118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:03.999734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b619a-8052-4aa3-a8b8-20060464a118 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:04.000543 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b619a-8052-4aa3-a8b8-20060464a118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:04.007340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b619a-8052-4aa3-a8b8-20060464a118 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:04.018467 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 991/1977] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:04.022270 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a8710-d1b4-43db-b9ff-2970c1ce3845 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:04.035997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a8710-d1b4-43db-b9ff-2970c1ce3845 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:04.043416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b64ae-889a-47f4-90d8-0221882f76d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:04.044070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b64ae-889a-47f4-90d8-0221882f76d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:04.044941 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b64ae-889a-47f4-90d8-0221882f76d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:04.045275 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b64ae-889a-47f4-90d8-0221882f76d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:04.046384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b64ae-889a-47f4-90d8-0221882f76d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:04.049050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b64ae-889a-47f4-90d8-0221882f76d6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:04.064821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 987/1978] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:04.071601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b619a-8052-4aa3-a8b8-20060464a118 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:04.083261 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b619a-8052-4aa3-a8b8-20060464a118 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:04.089910 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab296b5f-4647-45ec-9f6d-f76fefd95e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:04.090635 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab296b5f-4647-45ec-9f6d-f76fefd95e9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:04.090922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab296b5f-4647-45ec-9f6d-f76fefd95e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:04.091202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab296b5f-4647-45ec-9f6d-f76fefd95e9e admin admin] PATH_INFO: `/v3/projects/9cb2dc5621a645d8853b76441c3e018e/users/ccc30d896f184a12b30fc8705da46f95/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:04.110064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab296b5f-4647-45ec-9f6d-f76fefd95e9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb2dc5621a645d8853b76441c3e018e, user_id=ccc30d896f184a12b30fc8705da46f95, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:04.112923 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab296b5f-4647-45ec-9f6d-f76fefd95e9e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:04.134542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 992/1979] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:04 2022] PUT /identity/v3/projects/9cb2dc5621a645d8853b76441c3e018e/users/ccc30d896f184a12b30fc8705da46f95/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:04.145208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c8d15-ac82-4be7-acc4-5e08864f120c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:04.145386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c8d15-ac82-4be7-acc4-5e08864f120c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:04.145666 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c8d15-ac82-4be7-acc4-5e08864f120c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:04.180424 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-da1c8d15-ac82-4be7-acc4-5e08864f120c None None] MFA Rules not processed for user `ccc30d896f184a12b30fc8705da46f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:04.182042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-da1c8d15-ac82-4be7-acc4-5e08864f120c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:04.258155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 988/1980] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:04.269692 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab296b5f-4647-45ec-9f6d-f76fefd95e9e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:04.271596 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ab296b5f-4647-45ec-9f6d-f76fefd95e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:04.345482 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab296b5f-4647-45ec-9f6d-f76fefd95e9e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:04.349255 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b6973-8049-43a4-9004-793f8fca5d09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:04.349887 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b6973-8049-43a4-9004-793f8fca5d09 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:04.350165 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b6973-8049-43a4-9004-793f8fca5d09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:04.350434 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b6973-8049-43a4-9004-793f8fca5d09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:04.360746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b6973-8049-43a4-9004-793f8fca5d09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:04.362253 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b6973-8049-43a4-9004-793f8fca5d09 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:04.373722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 993/1981] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:05.951605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c8d15-ac82-4be7-acc4-5e08864f120c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:05.953989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-da1c8d15-ac82-4be7-acc4-5e08864f120c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:06.123269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c8d15-ac82-4be7-acc4-5e08864f120c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:06.131100 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b72a62-d596-4377-a40c-bc32436c1b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:06.131761 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b72a62-d596-4377-a40c-bc32436c1b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:06.133272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b72a62-d596-4377-a40c-bc32436c1b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:06.133272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b72a62-d596-4377-a40c-bc32436c1b6c admin admin] PATH_INFO: `/v3/users/48e839a4bc894e5b92fabc8c6f03c8a9` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:06.134454 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b72a62-d596-4377-a40c-bc32436c1b6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=48e839a4bc894e5b92fabc8c6f03c8a9)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:06.138431 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b72a62-d596-4377-a40c-bc32436c1b6c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:06.194614 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:06.257252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-49b72a62-d596-4377-a40c-bc32436c1b6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48e839a4bc894e5b92fabc8c6f03c8a9'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:06.258084 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-49b72a62-d596-4377-a40c-bc32436c1b6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48e839a4bc894e5b92fabc8c6f03c8a9'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:06.281569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-49b72a62-d596-4377-a40c-bc32436c1b6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48e839a4bc894e5b92fabc8c6f03c8a9'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:06.282410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-49b72a62-d596-4377-a40c-bc32436c1b6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48e839a4bc894e5b92fabc8c6f03c8a9'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:06.295250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-49b72a62-d596-4377-a40c-bc32436c1b6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48e839a4bc894e5b92fabc8c6f03c8a9'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:06.318595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 989/1982] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:05 2022] DELETE /identity/v3/users/48e839a4bc894e5b92fabc8c6f03c8a9 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:06.320920 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106e6d3-92a6-41d6-bf45-dfab08b59b8f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:06.321400 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106e6d3-92a6-41d6-bf45-dfab08b59b8f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:06.321400 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106e6d3-92a6-41d6-bf45-dfab08b59b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:06.357071 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e106e6d3-92a6-41d6-bf45-dfab08b59b8f None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:06.363729 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e106e6d3-92a6-41d6-bf45-dfab08b59b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:06.406702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:06.485595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 994/1983] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:56:06.498144 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b72a62-d596-4377-a40c-bc32436c1b6c admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:06.499771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-49b72a62-d596-4377-a40c-bc32436c1b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:06.582876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b72a62-d596-4377-a40c-bc32436c1b6c admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:06.589228 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1436a-801f-4b46-bfbf-19e79d5b271a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:06.590744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1436a-801f-4b46-bfbf-19e79d5b271a service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:06.591103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1436a-801f-4b46-bfbf-19e79d5b271a service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:06.591450 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1436a-801f-4b46-bfbf-19e79d5b271a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:06.604703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1436a-801f-4b46-bfbf-19e79d5b271a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:06.606320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1436a-801f-4b46-bfbf-19e79d5b271a service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:06.618357 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 990/1984] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:06.763986 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106e6d3-92a6-41d6-bf45-dfab08b59b8f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:06.765747 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e106e6d3-92a6-41d6-bf45-dfab08b59b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:06.780678 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106e6d3-92a6-41d6-bf45-dfab08b59b8f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:06.787798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3bcb2-7fa5-485a-818b-924f78a901e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:06.789077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3bcb2-7fa5-485a-818b-924f78a901e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:06.789484 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3bcb2-7fa5-485a-818b-924f78a901e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:06.789830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3bcb2-7fa5-485a-818b-924f78a901e2 admin admin] PATH_INFO: `/v3/users/e72ae4e2fa4541aa8b046ddba2bc0a3b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:06.791346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3bcb2-7fa5-485a-818b-924f78a901e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e72ae4e2fa4541aa8b046ddba2bc0a3b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:06.793395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3bcb2-7fa5-485a-818b-924f78a901e2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:06.870953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-68b3bcb2-7fa5-485a-818b-924f78a901e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e72ae4e2fa4541aa8b046ddba2bc0a3b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:06.886341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-68b3bcb2-7fa5-485a-818b-924f78a901e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e72ae4e2fa4541aa8b046ddba2bc0a3b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:06.895571 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-68b3bcb2-7fa5-485a-818b-924f78a901e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e72ae4e2fa4541aa8b046ddba2bc0a3b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:06.898435 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-68b3bcb2-7fa5-485a-818b-924f78a901e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e72ae4e2fa4541aa8b046ddba2bc0a3b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:06.906500 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-68b3bcb2-7fa5-485a-818b-924f78a901e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e72ae4e2fa4541aa8b046ddba2bc0a3b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:06.916121 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 995/1985] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:06 2022] DELETE /identity/v3/users/e72ae4e2fa4541aa8b046ddba2bc0a3b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:06.920119 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed0c9b-5f04-4bcc-82b6-6bd2d79b240c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:06.920566 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed0c9b-5f04-4bcc-82b6-6bd2d79b240c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:06.920844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed0c9b-5f04-4bcc-82b6-6bd2d79b240c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:06.941947 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-0eed0c9b-5f04-4bcc-82b6-6bd2d79b240c None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:06.943227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0eed0c9b-5f04-4bcc-82b6-6bd2d79b240c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:07.008975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 991/1986] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:56:07.021623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3bcb2-7fa5-485a-818b-924f78a901e2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:07.023806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-68b3bcb2-7fa5-485a-818b-924f78a901e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:07.062440 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed0c9b-5f04-4bcc-82b6-6bd2d79b240c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:07.063911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0eed0c9b-5f04-4bcc-82b6-6bd2d79b240c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:07.095403 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed0c9b-5f04-4bcc-82b6-6bd2d79b240c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:07.097604 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68516115-f0a1-40a4-9e85-4a427dd7e56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:07.098215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68516115-f0a1-40a4-9e85-4a427dd7e56e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:07.098542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68516115-f0a1-40a4-9e85-4a427dd7e56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:07.098762 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68516115-f0a1-40a4-9e85-4a427dd7e56e admin admin] PATH_INFO: `/v3/projects/1d9a526a38394b429f07b6ab4f80ebaa` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:07.099903 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68516115-f0a1-40a4-9e85-4a427dd7e56e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d9a526a38394b429f07b6ab4f80ebaa)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:07.102589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68516115-f0a1-40a4-9e85-4a427dd7e56e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:07.110542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3bcb2-7fa5-485a-818b-924f78a901e2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:07.114199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f1fea-a21b-4685-9eac-e8caf1f6dece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:07.114803 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f1fea-a21b-4685-9eac-e8caf1f6dece service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:07.115081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f1fea-a21b-4685-9eac-e8caf1f6dece service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:07.115358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f1fea-a21b-4685-9eac-e8caf1f6dece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:07.124892 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f1fea-a21b-4685-9eac-e8caf1f6dece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:07.126318 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f1fea-a21b-4685-9eac-e8caf1f6dece service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:07.136428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 996/1987] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:07.142685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:07.154760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-68516115-f0a1-40a4-9e85-4a427dd7e56e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d9a526a38394b429f07b6ab4f80ebaa', 'request_id': 'req-68516115-f0a1-40a4-9e85-4a427dd7e56e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:07.160377 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-68516115-f0a1-40a4-9e85-4a427dd7e56e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d9a526a38394b429f07b6ab4f80ebaa', 'request_id': 'req-68516115-f0a1-40a4-9e85-4a427dd7e56e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:07.167914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-68516115-f0a1-40a4-9e85-4a427dd7e56e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d9a526a38394b429f07b6ab4f80ebaa', 'request_id': 'req-68516115-f0a1-40a4-9e85-4a427dd7e56e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:07.182173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-68516115-f0a1-40a4-9e85-4a427dd7e56e admin admin] The token cache is being invalidate because project 1d9a526a38394b429f07b6ab4f80ebaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:07.182465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-68516115-f0a1-40a4-9e85-4a427dd7e56e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:07.183118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-68516115-f0a1-40a4-9e85-4a427dd7e56e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:07.184860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 992/1988] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:07 2022] DELETE /identity/v3/projects/1d9a526a38394b429f07b6ab4f80ebaa => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:07.189221 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f1fea-a21b-4685-9eac-e8caf1f6dece service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:07.200720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f1fea-a21b-4685-9eac-e8caf1f6dece service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:07.247080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:07.261886 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e929e-1537-44b9-b5dc-48408086b2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:07.262523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e929e-1537-44b9-b5dc-48408086b2c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:07.262801 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e929e-1537-44b9-b5dc-48408086b2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:07.263071 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e929e-1537-44b9-b5dc-48408086b2c2 admin admin] PATH_INFO: `/v3/users/ee955e16f37945cc92c123f1ff3aef45` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:07.264280 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e929e-1537-44b9-b5dc-48408086b2c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee955e16f37945cc92c123f1ff3aef45)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:07.266195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e929e-1537-44b9-b5dc-48408086b2c2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:07.323401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-168e929e-1537-44b9-b5dc-48408086b2c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee955e16f37945cc92c123f1ff3aef45'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:07.336130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-168e929e-1537-44b9-b5dc-48408086b2c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee955e16f37945cc92c123f1ff3aef45'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:07.343985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-168e929e-1537-44b9-b5dc-48408086b2c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee955e16f37945cc92c123f1ff3aef45'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:07.345094 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-168e929e-1537-44b9-b5dc-48408086b2c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee955e16f37945cc92c123f1ff3aef45'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:07.352507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-168e929e-1537-44b9-b5dc-48408086b2c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee955e16f37945cc92c123f1ff3aef45'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:07.359042 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 997/1989] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:07 2022] DELETE /identity/v3/users/ee955e16f37945cc92c123f1ff3aef45 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:07.437475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68516115-f0a1-40a4-9e85-4a427dd7e56e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:07.439446 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-68516115-f0a1-40a4-9e85-4a427dd7e56e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:07.455144 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68516115-f0a1-40a4-9e85-4a427dd7e56e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:07.526387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f2844-16c7-4782-a782-2d19dee84d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:07.527317 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f2844-16c7-4782-a782-2d19dee84d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:07.527596 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f2844-16c7-4782-a782-2d19dee84d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:07.527862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f2844-16c7-4782-a782-2d19dee84d0f admin admin] PATH_INFO: `/v3/projects/8e646215bae9482d9b1e00b35dd8b20e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:07.529046 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f2844-16c7-4782-a782-2d19dee84d0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e646215bae9482d9b1e00b35dd8b20e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:07.531073 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f2844-16c7-4782-a782-2d19dee84d0f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:07.605316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1d3f2844-16c7-4782-a782-2d19dee84d0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e646215bae9482d9b1e00b35dd8b20e', 'request_id': 'req-1d3f2844-16c7-4782-a782-2d19dee84d0f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:07.612418 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1d3f2844-16c7-4782-a782-2d19dee84d0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e646215bae9482d9b1e00b35dd8b20e', 'request_id': 'req-1d3f2844-16c7-4782-a782-2d19dee84d0f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:07.619994 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1d3f2844-16c7-4782-a782-2d19dee84d0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e646215bae9482d9b1e00b35dd8b20e', 'request_id': 'req-1d3f2844-16c7-4782-a782-2d19dee84d0f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:07.634840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1d3f2844-16c7-4782-a782-2d19dee84d0f admin admin] The token cache is being invalidate because project 8e646215bae9482d9b1e00b35dd8b20e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:07.635194 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1d3f2844-16c7-4782-a782-2d19dee84d0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:07.635832 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1d3f2844-16c7-4782-a782-2d19dee84d0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:07.637698 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 993/1990] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:07 2022] DELETE /identity/v3/projects/8e646215bae9482d9b1e00b35dd8b20e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:07.642542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118bd7c-6ad1-4655-a5ab-62a982db652d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:07.642834 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118bd7c-6ad1-4655-a5ab-62a982db652d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:07.643109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118bd7c-6ad1-4655-a5ab-62a982db652d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:07.664488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5118bd7c-6ad1-4655-a5ab-62a982db652d None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:07.665973 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5118bd7c-6ad1-4655-a5ab-62a982db652d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:07.743974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 998/1991] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:07.752823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05daaa-6f25-4383-8e29-f250ce5d11a9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:07.753301 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05daaa-6f25-4383-8e29-f250ce5d11a9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:07.753603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05daaa-6f25-4383-8e29-f250ce5d11a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:07.774937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-6c05daaa-6f25-4383-8e29-f250ce5d11a9 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:07.776239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6c05daaa-6f25-4383-8e29-f250ce5d11a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:07.784556 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 994/1992] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:07.788941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118bd7c-6ad1-4655-a5ab-62a982db652d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:07.802871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118bd7c-6ad1-4655-a5ab-62a982db652d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:07.809862 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5983157-fb0a-470c-b33e-8ec823aac570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:07.810616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5983157-fb0a-470c-b33e-8ec823aac570 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:07.810913 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5983157-fb0a-470c-b33e-8ec823aac570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:07.811187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5983157-fb0a-470c-b33e-8ec823aac570 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:07.811987 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5983157-fb0a-470c-b33e-8ec823aac570 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:56:07.812330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5983157-fb0a-470c-b33e-8ec823aac570 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:07.813938 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5983157-fb0a-470c-b33e-8ec823aac570 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:07.828114 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 999/1993] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:56:07.832406 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05daaa-6f25-4383-8e29-f250ce5d11a9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:07.844267 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05daaa-6f25-4383-8e29-f250ce5d11a9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:07.849775 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1708b0-0c46-4f1f-9a60-9d169ffe7ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:07.850673 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1708b0-0c46-4f1f-9a60-9d169ffe7ccb admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:07.851480 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1708b0-0c46-4f1f-9a60-9d169ffe7ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:07.851905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1708b0-0c46-4f1f-9a60-9d169ffe7ccb admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:07.852803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1708b0-0c46-4f1f-9a60-9d169ffe7ccb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:07.855173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1708b0-0c46-4f1f-9a60-9d169ffe7ccb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:07.892881 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 995/1994] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:07 2022] POST /identity/v3/projects => generated 399 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:56:07.900538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5983157-fb0a-470c-b33e-8ec823aac570 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:07.940301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5983157-fb0a-470c-b33e-8ec823aac570 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:07.940680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1708b0-0c46-4f1f-9a60-9d169ffe7ccb admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:07.940680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ce1708b0-0c46-4f1f-9a60-9d169ffe7ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:07.996640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:07.999902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:08.009106 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77f845-ebd5-41fc-91d0-ed087672fbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:08.009795 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77f845-ebd5-41fc-91d0-ed087672fbe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.010100 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77f845-ebd5-41fc-91d0-ed087672fbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.010545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77f845-ebd5-41fc-91d0-ed087672fbe9 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.011327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77f845-ebd5-41fc-91d0-ed087672fbe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:08.014995 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77f845-ebd5-41fc-91d0-ed087672fbe9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:08.044857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1708b0-0c46-4f1f-9a60-9d169ffe7ccb admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:08.045320 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1000/1995] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:07 2022] POST /identity/v3/users => generated 300 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:08.049113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77f845-ebd5-41fc-91d0-ed087672fbe9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:08.050859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a0668-8a87-4010-90e1-8a5f9971d100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:08.051596 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a0668-8a87-4010-90e1-8a5f9971d100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.051979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a0668-8a87-4010-90e1-8a5f9971d100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.052361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a0668-8a87-4010-90e1-8a5f9971d100 admin admin] PATH_INFO: `/v3/projects/f6f89f9594ab444e8c4b5847b9ce5473` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.053664 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a0668-8a87-4010-90e1-8a5f9971d100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6f89f9594ab444e8c4b5847b9ce5473)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:08.055979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a0668-8a87-4010-90e1-8a5f9971d100 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:08.059842 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77f845-ebd5-41fc-91d0-ed087672fbe9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:08.072372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edafb2-64be-4735-89e5-77bd422305ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:08.072372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edafb2-64be-4735-89e5-77bd422305ac admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.072372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edafb2-64be-4735-89e5-77bd422305ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.072372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edafb2-64be-4735-89e5-77bd422305ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.072725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edafb2-64be-4735-89e5-77bd422305ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:08.072725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edafb2-64be-4735-89e5-77bd422305ac admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:08.084902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1001/1996] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:56:08.089292 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edafb2-64be-4735-89e5-77bd422305ac admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:08.107723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edafb2-64be-4735-89e5-77bd422305ac admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:08.129911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-aa1a0668-8a87-4010-90e1-8a5f9971d100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6f89f9594ab444e8c4b5847b9ce5473', 'request_id': 'req-aa1a0668-8a87-4010-90e1-8a5f9971d100'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:08.137638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-aa1a0668-8a87-4010-90e1-8a5f9971d100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6f89f9594ab444e8c4b5847b9ce5473', 'request_id': 'req-aa1a0668-8a87-4010-90e1-8a5f9971d100'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:08.145380 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-aa1a0668-8a87-4010-90e1-8a5f9971d100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6f89f9594ab444e8c4b5847b9ce5473', 'request_id': 'req-aa1a0668-8a87-4010-90e1-8a5f9971d100'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:08.161890 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-aa1a0668-8a87-4010-90e1-8a5f9971d100 admin admin] The token cache is being invalidate because project f6f89f9594ab444e8c4b5847b9ce5473 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:08.163177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-aa1a0668-8a87-4010-90e1-8a5f9971d100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:08.164048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-aa1a0668-8a87-4010-90e1-8a5f9971d100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:08.166941 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 996/1997] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:07 2022] DELETE /identity/v3/projects/f6f89f9594ab444e8c4b5847b9ce5473 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:08.173375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa35698-f066-4841-81b9-30362d4b6ed0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.173888 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa35698-f066-4841-81b9-30362d4b6ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.174329 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa35698-f066-4841-81b9-30362d4b6ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.189961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba45dd6-a970-43fe-9643-fbd64ebc5ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:08.190762 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba45dd6-a970-43fe-9643-fbd64ebc5ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.191206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba45dd6-a970-43fe-9643-fbd64ebc5ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.191879 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba45dd6-a970-43fe-9643-fbd64ebc5ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.192842 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba45dd6-a970-43fe-9643-fbd64ebc5ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:08.195202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba45dd6-a970-43fe-9643-fbd64ebc5ce8 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:08.198515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-8aa35698-f066-4841-81b9-30362d4b6ed0 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:08.199738 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8aa35698-f066-4841-81b9-30362d4b6ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:08.216233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1002/1998] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:08.216416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 997/1999] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:56:08.217981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba45dd6-a970-43fe-9643-fbd64ebc5ce8 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:08.218153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d790e-dce9-4903-89ff-5b1e5d350554 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.218439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d790e-dce9-4903-89ff-5b1e5d350554 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.218715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d790e-dce9-4903-89ff-5b1e5d350554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.221516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6ba45dd6-a970-43fe-9643-fbd64ebc5ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:08.234486 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba45dd6-a970-43fe-9643-fbd64ebc5ce8 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:08.239064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b38d790e-dce9-4903-89ff-5b1e5d350554 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:08.239383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460173ef-1aed-423e-a7c2-db0f8ff1964a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:08.240039 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460173ef-1aed-423e-a7c2-db0f8ff1964a admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.240290 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b38d790e-dce9-4903-89ff-5b1e5d350554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:08.240430 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460173ef-1aed-423e-a7c2-db0f8ff1964a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.240777 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460173ef-1aed-423e-a7c2-db0f8ff1964a admin admin] PATH_INFO: `/v3/projects/686f07a065834096a008f8839f220d15/users/694a1a7671c540aeb3fddba18cf67e13/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.249968 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 998/2000] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:56:08.255592 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d790e-dce9-4903-89ff-5b1e5d350554 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:08.258091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460173ef-1aed-423e-a7c2-db0f8ff1964a admin admin] RBAC: Authorizing `identity:create_grant(project_id=686f07a065834096a008f8839f220d15, user_id=694a1a7671c540aeb3fddba18cf67e13, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:08.260785 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460173ef-1aed-423e-a7c2-db0f8ff1964a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:08.265776 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d790e-dce9-4903-89ff-5b1e5d350554 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:08.270661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0b2f1-a3f7-432b-bec5-ae4b8f3b2485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:08.271263 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0b2f1-a3f7-432b-bec5-ae4b8f3b2485 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.271535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0b2f1-a3f7-432b-bec5-ae4b8f3b2485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.271854 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0b2f1-a3f7-432b-bec5-ae4b8f3b2485 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.272557 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d0b2f1-a3f7-432b-bec5-ae4b8f3b2485 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:56:08.273439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d0b2f1-a3f7-432b-bec5-ae4b8f3b2485 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:08.275057 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1003/2001] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:08 2022] PUT /identity/v3/projects/686f07a065834096a008f8839f220d15/users/694a1a7671c540aeb3fddba18cf67e13/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:56:08.276353 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d0b2f1-a3f7-432b-bec5-ae4b8f3b2485 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:08.285615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f83841-e64b-42a4-8f80-dcf0170d8311 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.285902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f83841-e64b-42a4-8f80-dcf0170d8311 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.286179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f83841-e64b-42a4-8f80-dcf0170d8311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.289520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 999/2002] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:56:08.293577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0b2f1-a3f7-432b-bec5-ae4b8f3b2485 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:08.303030 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0b2f1-a3f7-432b-bec5-ae4b8f3b2485 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:08.314579 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-52f83841-e64b-42a4-8f80-dcf0170d8311 None None] MFA Rules not processed for user `694a1a7671c540aeb3fddba18cf67e13`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:08.315778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-52f83841-e64b-42a4-8f80-dcf0170d8311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:08.371345 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379b1bb-fc83-43b9-9f9c-18935637c5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:08.371990 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379b1bb-fc83-43b9-9f9c-18935637c5e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.372299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379b1bb-fc83-43b9-9f9c-18935637c5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.372683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379b1bb-fc83-43b9-9f9c-18935637c5e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.373342 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379b1bb-fc83-43b9-9f9c-18935637c5e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:08.375267 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379b1bb-fc83-43b9-9f9c-18935637c5e3 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:08.390196 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1004/2003] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:08 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:56:08.402625 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f83841-e64b-42a4-8f80-dcf0170d8311 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:08.404024 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-52f83841-e64b-42a4-8f80-dcf0170d8311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:08.407979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1000/2004] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:08 2022] POST /identity/v3/projects => generated 363 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:08.409041 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379b1bb-fc83-43b9-9f9c-18935637c5e3 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:08.428053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:08.440825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:08.465407 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f83841-e64b-42a4-8f80-dcf0170d8311 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:08.475948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379b1bb-fc83-43b9-9f9c-18935637c5e3 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:08.480831 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3d93e-f8fb-4fff-a894-5f2642240342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:08.481434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3d93e-f8fb-4fff-a894-5f2642240342 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.481716 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3d93e-f8fb-4fff-a894-5f2642240342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.482347 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3d93e-f8fb-4fff-a894-5f2642240342 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.482604 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3d93e-f8fb-4fff-a894-5f2642240342 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:08.484556 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3d93e-f8fb-4fff-a894-5f2642240342 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:08.509764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1001/2005] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:08 2022] POST /identity/v3/users => generated 282 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:56:08.510791 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3d93e-f8fb-4fff-a894-5f2642240342 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:08.516746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077d587-7cc9-4f05-bd42-8303ad623742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:08.517361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077d587-7cc9-4f05-bd42-8303ad623742 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.517639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077d587-7cc9-4f05-bd42-8303ad623742 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.517938 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077d587-7cc9-4f05-bd42-8303ad623742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.520453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3d93e-f8fb-4fff-a894-5f2642240342 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:08.525052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502ac56-22d1-4a54-aa6c-00585160b637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:08.525629 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502ac56-22d1-4a54-aa6c-00585160b637 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.525903 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502ac56-22d1-4a54-aa6c-00585160b637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.526172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502ac56-22d1-4a54-aa6c-00585160b637 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.526802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502ac56-22d1-4a54-aa6c-00585160b637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:08.528403 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502ac56-22d1-4a54-aa6c-00585160b637 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:08.538512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1002/2006] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:56:08.542393 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502ac56-22d1-4a54-aa6c-00585160b637 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:08.551817 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502ac56-22d1-4a54-aa6c-00585160b637 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:08.556952 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d8dd8-b3c7-406c-aafe-86ff50c1ffcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:08.557516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d8dd8-b3c7-406c-aafe-86ff50c1ffcc admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.557791 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d8dd8-b3c7-406c-aafe-86ff50c1ffcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.558062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d8dd8-b3c7-406c-aafe-86ff50c1ffcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.558671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d8dd8-b3c7-406c-aafe-86ff50c1ffcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:08.560275 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d8dd8-b3c7-406c-aafe-86ff50c1ffcc admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:08.570775 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1003/2007] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:56:08.574921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d8dd8-b3c7-406c-aafe-86ff50c1ffcc admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:08.576664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077d587-7cc9-4f05-bd42-8303ad623742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:08.578276 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077d587-7cc9-4f05-bd42-8303ad623742 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:08.584356 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d8dd8-b3c7-406c-aafe-86ff50c1ffcc admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:08.587728 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1005/2008] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 07:56:08.589679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1950e77-3587-4fa7-ab63-98181404a8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:08.590318 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1950e77-3587-4fa7-ab63-98181404a8f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.590598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1950e77-3587-4fa7-ab63-98181404a8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.590869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1950e77-3587-4fa7-ab63-98181404a8f5 admin admin] PATH_INFO: `/v3/projects/4876ec1d3559442484595f10e412ca58/users/30d69b9ce0214049a029489b11e719b5/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.606245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1950e77-3587-4fa7-ab63-98181404a8f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4876ec1d3559442484595f10e412ca58, user_id=30d69b9ce0214049a029489b11e719b5, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:08.608932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1950e77-3587-4fa7-ab63-98181404a8f5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:08.623152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1004/2009] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:08 2022] PUT /identity/v3/projects/4876ec1d3559442484595f10e412ca58/users/30d69b9ce0214049a029489b11e719b5/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:08.634704 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1c62d-d46d-4477-a483-f1c3497aafbf None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.634991 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1c62d-d46d-4477-a483-f1c3497aafbf None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.635267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1c62d-d46d-4477-a483-f1c3497aafbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.664016 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-55d1c62d-d46d-4477-a483-f1c3497aafbf None None] MFA Rules not processed for user `30d69b9ce0214049a029489b11e719b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:08.665273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-55d1c62d-d46d-4477-a483-f1c3497aafbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:08.748755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1006/2010] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:08.761758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1950e77-3587-4fa7-ab63-98181404a8f5 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:08.823514 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1950e77-3587-4fa7-ab63-98181404a8f5 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:08.827422 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3d7d5-f48b-4003-951c-e44bfce39ec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:08.828565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3d7d5-f48b-4003-951c-e44bfce39ec1 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:08.828850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3d7d5-f48b-4003-951c-e44bfce39ec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:08.829123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3d7d5-f48b-4003-951c-e44bfce39ec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:08.839215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3d7d5-f48b-4003-951c-e44bfce39ec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:08.840751 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3d7d5-f48b-4003-951c-e44bfce39ec1 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:08.853534 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1005/2011] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:11.230114 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1c62d-d46d-4477-a483-f1c3497aafbf None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:11.231676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-55d1c62d-d46d-4477-a483-f1c3497aafbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:11.276499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:11.309503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1c62d-d46d-4477-a483-f1c3497aafbf None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:11.314099 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060b255-7f90-4b9b-adbe-7c2942b30782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:11.314726 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060b255-7f90-4b9b-adbe-7c2942b30782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:11.315003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060b255-7f90-4b9b-adbe-7c2942b30782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:11.315275 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060b255-7f90-4b9b-adbe-7c2942b30782 admin admin] PATH_INFO: `/v3/users/d13d5bb983d749cdbef7dcb2a18b0fd5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:11.316693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060b255-7f90-4b9b-adbe-7c2942b30782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d13d5bb983d749cdbef7dcb2a18b0fd5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:11.318588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060b255-7f90-4b9b-adbe-7c2942b30782 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:11.363595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7060b255-7f90-4b9b-adbe-7c2942b30782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd13d5bb983d749cdbef7dcb2a18b0fd5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:11.375689 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7060b255-7f90-4b9b-adbe-7c2942b30782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd13d5bb983d749cdbef7dcb2a18b0fd5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:11.384404 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7060b255-7f90-4b9b-adbe-7c2942b30782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd13d5bb983d749cdbef7dcb2a18b0fd5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:11.385035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7060b255-7f90-4b9b-adbe-7c2942b30782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd13d5bb983d749cdbef7dcb2a18b0fd5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:11.392519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7060b255-7f90-4b9b-adbe-7c2942b30782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd13d5bb983d749cdbef7dcb2a18b0fd5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:11.398442 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1007/2012] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:11 2022] DELETE /identity/v3/users/d13d5bb983d749cdbef7dcb2a18b0fd5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:11.402939 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37394529-c55d-44ba-9dd1-0207649ff10f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:11.403385 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37394529-c55d-44ba-9dd1-0207649ff10f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:11.403475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37394529-c55d-44ba-9dd1-0207649ff10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:11.422270 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-37394529-c55d-44ba-9dd1-0207649ff10f None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:11.423470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-37394529-c55d-44ba-9dd1-0207649ff10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:11.484260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1006/2013] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:11.494715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060b255-7f90-4b9b-adbe-7c2942b30782 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:11.496160 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7060b255-7f90-4b9b-adbe-7c2942b30782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:11.554588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060b255-7f90-4b9b-adbe-7c2942b30782 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:11.558451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a4b24-7dd3-440e-86cb-cb7e64ff7384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:11.559052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a4b24-7dd3-440e-86cb-cb7e64ff7384 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:11.559332 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a4b24-7dd3-440e-86cb-cb7e64ff7384 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:11.559607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a4b24-7dd3-440e-86cb-cb7e64ff7384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:11.569264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a4b24-7dd3-440e-86cb-cb7e64ff7384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:11.571679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a4b24-7dd3-440e-86cb-cb7e64ff7384 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:11.581512 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1008/2014] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:11.863944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37394529-c55d-44ba-9dd1-0207649ff10f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:11.865566 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-37394529-c55d-44ba-9dd1-0207649ff10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:11.873308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:11.880505 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37394529-c55d-44ba-9dd1-0207649ff10f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:11.886527 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425bde7-834e-4c2a-ad28-7aa19bf762a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:11.887162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425bde7-834e-4c2a-ad28-7aa19bf762a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:11.887443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425bde7-834e-4c2a-ad28-7aa19bf762a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:11.887742 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425bde7-834e-4c2a-ad28-7aa19bf762a7 admin admin] PATH_INFO: `/v3/projects/045df47d901b4fb3a0f7ad9cc8e0f9c2` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:11.890243 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425bde7-834e-4c2a-ad28-7aa19bf762a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=045df47d901b4fb3a0f7ad9cc8e0f9c2)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:11.892848 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425bde7-834e-4c2a-ad28-7aa19bf762a7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:11.952596 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a425bde7-834e-4c2a-ad28-7aa19bf762a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '045df47d901b4fb3a0f7ad9cc8e0f9c2', 'request_id': 'req-a425bde7-834e-4c2a-ad28-7aa19bf762a7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:11.958863 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a425bde7-834e-4c2a-ad28-7aa19bf762a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '045df47d901b4fb3a0f7ad9cc8e0f9c2', 'request_id': 'req-a425bde7-834e-4c2a-ad28-7aa19bf762a7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:11.969346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a425bde7-834e-4c2a-ad28-7aa19bf762a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '045df47d901b4fb3a0f7ad9cc8e0f9c2', 'request_id': 'req-a425bde7-834e-4c2a-ad28-7aa19bf762a7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:11.985945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a425bde7-834e-4c2a-ad28-7aa19bf762a7 admin admin] The token cache is being invalidate because project 045df47d901b4fb3a0f7ad9cc8e0f9c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:11.986236 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a425bde7-834e-4c2a-ad28-7aa19bf762a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:11.986927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a425bde7-834e-4c2a-ad28-7aa19bf762a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:11.989016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1007/2015] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:11 2022] DELETE /identity/v3/projects/045df47d901b4fb3a0f7ad9cc8e0f9c2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:11.995256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a4b24-7dd3-440e-86cb-cb7e64ff7384 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:12.006266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a4b24-7dd3-440e-86cb-cb7e64ff7384 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:12.067329 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:12.076470 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4ef51-0f02-4182-ac42-4d7a129f3540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:12.077407 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4ef51-0f02-4182-ac42-4d7a129f3540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:12.077866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4ef51-0f02-4182-ac42-4d7a129f3540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:12.078296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4ef51-0f02-4182-ac42-4d7a129f3540 admin admin] PATH_INFO: `/v3/users/e07e00059684444794bf17a766fe16f0` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:12.079766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4ef51-0f02-4182-ac42-4d7a129f3540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e07e00059684444794bf17a766fe16f0)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:12.081838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4ef51-0f02-4182-ac42-4d7a129f3540 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:12.140509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b3a4ef51-0f02-4182-ac42-4d7a129f3540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e07e00059684444794bf17a766fe16f0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:12.155475 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b3a4ef51-0f02-4182-ac42-4d7a129f3540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e07e00059684444794bf17a766fe16f0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:12.165465 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b3a4ef51-0f02-4182-ac42-4d7a129f3540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e07e00059684444794bf17a766fe16f0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:12.166105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b3a4ef51-0f02-4182-ac42-4d7a129f3540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e07e00059684444794bf17a766fe16f0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:12.179580 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b3a4ef51-0f02-4182-ac42-4d7a129f3540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e07e00059684444794bf17a766fe16f0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:12.186208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1009/2016] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:11 2022] DELETE /identity/v3/users/e07e00059684444794bf17a766fe16f0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:12.487457 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425bde7-834e-4c2a-ad28-7aa19bf762a7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:12.488975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a425bde7-834e-4c2a-ad28-7aa19bf762a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:12.504240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425bde7-834e-4c2a-ad28-7aa19bf762a7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:12.580385 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc3c50-3321-4b2d-b1f8-ba439a4938e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:12.581024 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc3c50-3321-4b2d-b1f8-ba439a4938e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:12.581300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc3c50-3321-4b2d-b1f8-ba439a4938e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:12.581565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc3c50-3321-4b2d-b1f8-ba439a4938e2 admin admin] PATH_INFO: `/v3/projects/eb00ba984ba64798893686720da1e14e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:12.582703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc3c50-3321-4b2d-b1f8-ba439a4938e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb00ba984ba64798893686720da1e14e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:12.584680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc3c50-3321-4b2d-b1f8-ba439a4938e2 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:12.635588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-46fc3c50-3321-4b2d-b1f8-ba439a4938e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb00ba984ba64798893686720da1e14e', 'request_id': 'req-46fc3c50-3321-4b2d-b1f8-ba439a4938e2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:12.641434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-46fc3c50-3321-4b2d-b1f8-ba439a4938e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb00ba984ba64798893686720da1e14e', 'request_id': 'req-46fc3c50-3321-4b2d-b1f8-ba439a4938e2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:12.649181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-46fc3c50-3321-4b2d-b1f8-ba439a4938e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb00ba984ba64798893686720da1e14e', 'request_id': 'req-46fc3c50-3321-4b2d-b1f8-ba439a4938e2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:12.662147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-46fc3c50-3321-4b2d-b1f8-ba439a4938e2 admin admin] The token cache is being invalidate because project eb00ba984ba64798893686720da1e14e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:12.662675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-46fc3c50-3321-4b2d-b1f8-ba439a4938e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:12.663615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-46fc3c50-3321-4b2d-b1f8-ba439a4938e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:12.665760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1008/2017] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:12 2022] DELETE /identity/v3/projects/eb00ba984ba64798893686720da1e14e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:12.672849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8c6b4-e03f-4b82-b7ff-abc0e22503ee None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:12.673330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8c6b4-e03f-4b82-b7ff-abc0e22503ee None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:12.673787 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8c6b4-e03f-4b82-b7ff-abc0e22503ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:12.694887 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a5f8c6b4-e03f-4b82-b7ff-abc0e22503ee None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:12.696369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a5f8c6b4-e03f-4b82-b7ff-abc0e22503ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:12.767668 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1010/2018] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:12.776485 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344154be-1af5-47d0-b110-9e833d86a062 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:12.776758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344154be-1af5-47d0-b110-9e833d86a062 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:12.777025 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344154be-1af5-47d0-b110-9e833d86a062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:12.806975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-344154be-1af5-47d0-b110-9e833d86a062 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:12.808566 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-344154be-1af5-47d0-b110-9e833d86a062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:12.817552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1009/2019] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:12.823606 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8c6b4-e03f-4b82-b7ff-abc0e22503ee None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:12.834422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8c6b4-e03f-4b82-b7ff-abc0e22503ee None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:12.839298 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297fba5-fb6f-4908-b48e-563580855942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:12.839904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297fba5-fb6f-4908-b48e-563580855942 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:12.840213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297fba5-fb6f-4908-b48e-563580855942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:12.840487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297fba5-fb6f-4908-b48e-563580855942 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:12.841133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297fba5-fb6f-4908-b48e-563580855942 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:56:12.841455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297fba5-fb6f-4908-b48e-563580855942 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:12.843053 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297fba5-fb6f-4908-b48e-563580855942 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:12.855145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1011/2020] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:56:12.860537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344154be-1af5-47d0-b110-9e833d86a062 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:12.870486 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344154be-1af5-47d0-b110-9e833d86a062 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:12.875597 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa7a7a-ff77-40d9-9853-27da70f16794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:12.876251 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa7a7a-ff77-40d9-9853-27da70f16794 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:12.876530 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa7a7a-ff77-40d9-9853-27da70f16794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:12.876802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa7a7a-ff77-40d9-9853-27da70f16794 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:12.877468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa7a7a-ff77-40d9-9853-27da70f16794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:12.879413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa7a7a-ff77-40d9-9853-27da70f16794 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:12.885742 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:12.910886 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1010/2021] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:12 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:12.918863 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297fba5-fb6f-4908-b48e-563580855942 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:12.929816 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297fba5-fb6f-4908-b48e-563580855942 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:12.987656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df58c6f-089c-45fd-9baf-e64f8404c7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:12.988312 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df58c6f-089c-45fd-9baf-e64f8404c7fc admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:12.988619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df58c6f-089c-45fd-9baf-e64f8404c7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:12.988893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df58c6f-089c-45fd-9baf-e64f8404c7fc admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:12.989529 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df58c6f-089c-45fd-9baf-e64f8404c7fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:12.992025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df58c6f-089c-45fd-9baf-e64f8404c7fc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:13.001088 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:13.015498 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1012/2022] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:12 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:13.019915 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa7a7a-ff77-40d9-9853-27da70f16794 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:13.031272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa7a7a-ff77-40d9-9853-27da70f16794 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:13.036710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278d26f-8b59-4bad-8856-012fcfa08e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:13.037321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278d26f-8b59-4bad-8856-012fcfa08e7e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:13.037677 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278d26f-8b59-4bad-8856-012fcfa08e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:13.037972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278d26f-8b59-4bad-8856-012fcfa08e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:13.038713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278d26f-8b59-4bad-8856-012fcfa08e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:13.040418 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278d26f-8b59-4bad-8856-012fcfa08e7e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:13.051094 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1011/2023] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:13.055911 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df58c6f-089c-45fd-9baf-e64f8404c7fc admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:13.065872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df58c6f-089c-45fd-9baf-e64f8404c7fc admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:13.071448 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8227ba-3853-424f-923d-f5f92b6caae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:13.072036 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8227ba-3853-424f-923d-f5f92b6caae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:13.072460 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8227ba-3853-424f-923d-f5f92b6caae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:13.072732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8227ba-3853-424f-923d-f5f92b6caae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:13.073395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8227ba-3853-424f-923d-f5f92b6caae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:13.074984 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8227ba-3853-424f-923d-f5f92b6caae4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:13.086088 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1013/2024] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:13.090781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278d26f-8b59-4bad-8856-012fcfa08e7e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:13.101175 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278d26f-8b59-4bad-8856-012fcfa08e7e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:13.105758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fddab2-092f-4ef0-8a84-163598bd2bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:13.106410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fddab2-092f-4ef0-8a84-163598bd2bef admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:13.106680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fddab2-092f-4ef0-8a84-163598bd2bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:13.106947 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fddab2-092f-4ef0-8a84-163598bd2bef admin admin] PATH_INFO: `/v3/projects/f564fe9817ce4f57b20da85b644c7d9e/users/ff4f0aa7300f454f9d625a16462a91b3/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:13.123245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fddab2-092f-4ef0-8a84-163598bd2bef admin admin] RBAC: Authorizing `identity:create_grant(project_id=f564fe9817ce4f57b20da85b644c7d9e, user_id=ff4f0aa7300f454f9d625a16462a91b3, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:13.125687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fddab2-092f-4ef0-8a84-163598bd2bef admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:13.139352 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1012/2025] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:13 2022] PUT /identity/v3/projects/f564fe9817ce4f57b20da85b644c7d9e/users/ff4f0aa7300f454f9d625a16462a91b3/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:13.149697 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc6fc1-c47e-4141-b476-fc492bb1453c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:13.150062 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc6fc1-c47e-4141-b476-fc492bb1453c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:13.150442 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc6fc1-c47e-4141-b476-fc492bb1453c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:13.179640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-42cc6fc1-c47e-4141-b476-fc492bb1453c None None] MFA Rules not processed for user `ff4f0aa7300f454f9d625a16462a91b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:13.181062 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-42cc6fc1-c47e-4141-b476-fc492bb1453c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:13.293570 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1014/2026] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:13.298281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fddab2-092f-4ef0-8a84-163598bd2bef admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:13.379894 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fddab2-092f-4ef0-8a84-163598bd2bef admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:13.385078 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610febe8-5ba6-493f-8751-b130457ed537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:13.386086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610febe8-5ba6-493f-8751-b130457ed537 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:13.386499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610febe8-5ba6-493f-8751-b130457ed537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:13.386754 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610febe8-5ba6-493f-8751-b130457ed537 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:13.387540 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610febe8-5ba6-493f-8751-b130457ed537 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:13.389578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610febe8-5ba6-493f-8751-b130457ed537 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:13.422224 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1013/2027] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:13 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:13.426307 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc6fc1-c47e-4141-b476-fc492bb1453c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:13.438131 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc6fc1-c47e-4141-b476-fc492bb1453c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:13.514850 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa6820-1145-4e33-bf73-461a2ad7d965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:13.515509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa6820-1145-4e33-bf73-461a2ad7d965 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:13.515824 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa6820-1145-4e33-bf73-461a2ad7d965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:13.516107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa6820-1145-4e33-bf73-461a2ad7d965 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:13.516784 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa6820-1145-4e33-bf73-461a2ad7d965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:13.518717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa6820-1145-4e33-bf73-461a2ad7d965 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:13.551296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1015/2028] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:13 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:13.552547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610febe8-5ba6-493f-8751-b130457ed537 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:13.566103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610febe8-5ba6-493f-8751-b130457ed537 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:13.572905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3d616-f7b1-4d60-bcc5-d96c933e5fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:13.574001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3d616-f7b1-4d60-bcc5-d96c933e5fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:13.574499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3d616-f7b1-4d60-bcc5-d96c933e5fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:13.574970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3d616-f7b1-4d60-bcc5-d96c933e5fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:13.575982 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3d616-f7b1-4d60-bcc5-d96c933e5fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:13.580093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3d616-f7b1-4d60-bcc5-d96c933e5fb8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:13.600120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1014/2029] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:13.603294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa6820-1145-4e33-bf73-461a2ad7d965 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:13.613685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa6820-1145-4e33-bf73-461a2ad7d965 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:13.618537 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebbe38-a705-4435-aaa8-54087ac4c54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:13.619206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebbe38-a705-4435-aaa8-54087ac4c54e admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:13.619480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebbe38-a705-4435-aaa8-54087ac4c54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:13.619750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebbe38-a705-4435-aaa8-54087ac4c54e admin admin] PATH_INFO: `/v3/projects/6e7b1c76d7fe4c058e71e0491f57afd9/users/3efb9881cdf0416d86d03b646b2a06f3/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:13.635369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ebbe38-a705-4435-aaa8-54087ac4c54e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e7b1c76d7fe4c058e71e0491f57afd9, user_id=3efb9881cdf0416d86d03b646b2a06f3, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:13.638038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ebbe38-a705-4435-aaa8-54087ac4c54e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:13.652415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1016/2030] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:13 2022] PUT /identity/v3/projects/6e7b1c76d7fe4c058e71e0491f57afd9/users/3efb9881cdf0416d86d03b646b2a06f3/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:13.661217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3d616-f7b1-4d60-bcc5-d96c933e5fb8 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:13.672443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3d616-f7b1-4d60-bcc5-d96c933e5fb8 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:13.731152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4adb9b-70ce-41ff-a4fb-3bd2250c8ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:13.731753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4adb9b-70ce-41ff-a4fb-3bd2250c8ecb admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:13.732028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4adb9b-70ce-41ff-a4fb-3bd2250c8ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:13.732329 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4adb9b-70ce-41ff-a4fb-3bd2250c8ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:13.732961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4adb9b-70ce-41ff-a4fb-3bd2250c8ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:13.734542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4adb9b-70ce-41ff-a4fb-3bd2250c8ecb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:13.744237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1015/2031] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:13.748370 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebbe38-a705-4435-aaa8-54087ac4c54e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:13.758451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebbe38-a705-4435-aaa8-54087ac4c54e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:13.763017 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2e773-ac76-44c7-8ad7-e9cdb2e14837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:13.763657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2e773-ac76-44c7-8ad7-e9cdb2e14837 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:13.763931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2e773-ac76-44c7-8ad7-e9cdb2e14837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:13.764235 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2e773-ac76-44c7-8ad7-e9cdb2e14837 admin admin] PATH_INFO: `/v3/projects/6e7b1c76d7fe4c058e71e0491f57afd9/users/3efb9881cdf0416d86d03b646b2a06f3/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:13.766276 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2e773-ac76-44c7-8ad7-e9cdb2e14837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e7b1c76d7fe4c058e71e0491f57afd9, user_id=3efb9881cdf0416d86d03b646b2a06f3, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:13.768966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2e773-ac76-44c7-8ad7-e9cdb2e14837 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:13.779656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1017/2032] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:13 2022] PUT /identity/v3/projects/6e7b1c76d7fe4c058e71e0491f57afd9/users/3efb9881cdf0416d86d03b646b2a06f3/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:13.791362 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e38bd6-cf2b-4b92-8711-86ddb9c60edd None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:13.791583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e38bd6-cf2b-4b92-8711-86ddb9c60edd None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:13.791858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e38bd6-cf2b-4b92-8711-86ddb9c60edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:13.820260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b4e38bd6-cf2b-4b92-8711-86ddb9c60edd None None] MFA Rules not processed for user `3efb9881cdf0416d86d03b646b2a06f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:13.821484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b4e38bd6-cf2b-4b92-8711-86ddb9c60edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:13.831091 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:13.894149 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1016/2033] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:13.905910 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2e773-ac76-44c7-8ad7-e9cdb2e14837 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:13.907636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e2d2e773-ac76-44c7-8ad7-e9cdb2e14837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:13.936207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:13.978614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2e773-ac76-44c7-8ad7-e9cdb2e14837 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:13.984788 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5fad1-d772-436f-92bb-5bc607df308a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:13.985584 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5fad1-d772-436f-92bb-5bc607df308a service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:13.985999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5fad1-d772-436f-92bb-5bc607df308a service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:13.986374 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5fad1-d772-436f-92bb-5bc607df308a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:13.996789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5fad1-d772-436f-92bb-5bc607df308a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:13.998233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5fad1-d772-436f-92bb-5bc607df308a service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:14.009075 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1018/2034] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:14.039228 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e38bd6-cf2b-4b92-8711-86ddb9c60edd None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:14.106116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e38bd6-cf2b-4b92-8711-86ddb9c60edd None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:14.110971 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075255c-d3d2-40b8-b7b3-435dbda4c558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:14.111600 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075255c-d3d2-40b8-b7b3-435dbda4c558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:14.111878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075255c-d3d2-40b8-b7b3-435dbda4c558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:14.112183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075255c-d3d2-40b8-b7b3-435dbda4c558 admin admin] PATH_INFO: `/v3/users/ff4f0aa7300f454f9d625a16462a91b3` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:14.113329 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075255c-d3d2-40b8-b7b3-435dbda4c558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff4f0aa7300f454f9d625a16462a91b3)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:14.115264 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075255c-d3d2-40b8-b7b3-435dbda4c558 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:14.162537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5075255c-d3d2-40b8-b7b3-435dbda4c558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff4f0aa7300f454f9d625a16462a91b3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:14.163352 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5075255c-d3d2-40b8-b7b3-435dbda4c558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff4f0aa7300f454f9d625a16462a91b3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:14.175001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5075255c-d3d2-40b8-b7b3-435dbda4c558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff4f0aa7300f454f9d625a16462a91b3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:14.175633 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5075255c-d3d2-40b8-b7b3-435dbda4c558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff4f0aa7300f454f9d625a16462a91b3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:14.182673 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5075255c-d3d2-40b8-b7b3-435dbda4c558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff4f0aa7300f454f9d625a16462a91b3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:14.196898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1017/2035] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:14 2022] DELETE /identity/v3/users/ff4f0aa7300f454f9d625a16462a91b3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:14.201598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925e056-9574-40fd-88ef-384076223dbe None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:14.201991 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925e056-9574-40fd-88ef-384076223dbe None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:14.202272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925e056-9574-40fd-88ef-384076223dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:14.221620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a925e056-9574-40fd-88ef-384076223dbe None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:14.222811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a925e056-9574-40fd-88ef-384076223dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:14.282234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1019/2036] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:14.292118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075255c-d3d2-40b8-b7b3-435dbda4c558 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:14.293505 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5075255c-d3d2-40b8-b7b3-435dbda4c558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:14.326044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:14.354050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075255c-d3d2-40b8-b7b3-435dbda4c558 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:14.357609 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffa5b1-f723-4e85-a85b-f8e09774a1ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:14.358207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffa5b1-f723-4e85-a85b-f8e09774a1ce service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:14.358480 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffa5b1-f723-4e85-a85b-f8e09774a1ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:14.358750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffa5b1-f723-4e85-a85b-f8e09774a1ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:14.368507 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffa5b1-f723-4e85-a85b-f8e09774a1ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:14.369949 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffa5b1-f723-4e85-a85b-f8e09774a1ce service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:14.390866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1018/2037] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:14.677282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925e056-9574-40fd-88ef-384076223dbe None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:14.679341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a925e056-9574-40fd-88ef-384076223dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:14.692477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925e056-9574-40fd-88ef-384076223dbe None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:14.698429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f0ba6-b295-494d-8517-421bd3ee62d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:14.699076 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f0ba6-b295-494d-8517-421bd3ee62d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:14.699356 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f0ba6-b295-494d-8517-421bd3ee62d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:14.699647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f0ba6-b295-494d-8517-421bd3ee62d4 admin admin] PATH_INFO: `/v3/projects/f564fe9817ce4f57b20da85b644c7d9e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:14.700790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f0ba6-b295-494d-8517-421bd3ee62d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f564fe9817ce4f57b20da85b644c7d9e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:14.702683 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f0ba6-b295-494d-8517-421bd3ee62d4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:14.753859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c99f0ba6-b295-494d-8517-421bd3ee62d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f564fe9817ce4f57b20da85b644c7d9e', 'request_id': 'req-c99f0ba6-b295-494d-8517-421bd3ee62d4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:14.759615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c99f0ba6-b295-494d-8517-421bd3ee62d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f564fe9817ce4f57b20da85b644c7d9e', 'request_id': 'req-c99f0ba6-b295-494d-8517-421bd3ee62d4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:14.765838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c99f0ba6-b295-494d-8517-421bd3ee62d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f564fe9817ce4f57b20da85b644c7d9e', 'request_id': 'req-c99f0ba6-b295-494d-8517-421bd3ee62d4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:14.779189 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c99f0ba6-b295-494d-8517-421bd3ee62d4 admin admin] The token cache is being invalidate because project f564fe9817ce4f57b20da85b644c7d9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:14.779478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c99f0ba6-b295-494d-8517-421bd3ee62d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:14.780164 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c99f0ba6-b295-494d-8517-421bd3ee62d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:14.781668 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1020/2038] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:14 2022] DELETE /identity/v3/projects/f564fe9817ce4f57b20da85b644c7d9e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:56:14.785656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffa5b1-f723-4e85-a85b-f8e09774a1ce service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:14.796335 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffa5b1-f723-4e85-a85b-f8e09774a1ce service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:14.855117 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb33442-fba1-4e11-897b-d0f792a035fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:14.855749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb33442-fba1-4e11-897b-d0f792a035fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:14.856059 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb33442-fba1-4e11-897b-d0f792a035fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:14.856352 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb33442-fba1-4e11-897b-d0f792a035fc admin admin] PATH_INFO: `/v3/users/3efb9881cdf0416d86d03b646b2a06f3` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:14.857516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb33442-fba1-4e11-897b-d0f792a035fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3efb9881cdf0416d86d03b646b2a06f3)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:14.859472 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb33442-fba1-4e11-897b-d0f792a035fc admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:14.905414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8eb33442-fba1-4e11-897b-d0f792a035fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3efb9881cdf0416d86d03b646b2a06f3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:14.906073 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8eb33442-fba1-4e11-897b-d0f792a035fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3efb9881cdf0416d86d03b646b2a06f3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:14.918266 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8eb33442-fba1-4e11-897b-d0f792a035fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3efb9881cdf0416d86d03b646b2a06f3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:14.918897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8eb33442-fba1-4e11-897b-d0f792a035fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3efb9881cdf0416d86d03b646b2a06f3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:14.925653 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8eb33442-fba1-4e11-897b-d0f792a035fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3efb9881cdf0416d86d03b646b2a06f3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:14.938638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1019/2039] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:14 2022] DELETE /identity/v3/users/3efb9881cdf0416d86d03b646b2a06f3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:15.279325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f0ba6-b295-494d-8517-421bd3ee62d4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:15.280769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c99f0ba6-b295-494d-8517-421bd3ee62d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:15.294335 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f0ba6-b295-494d-8517-421bd3ee62d4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:15.354922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:15.368940 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b9249-8ad8-40d9-a7a3-f9bc60c8cd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:15.369573 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b9249-8ad8-40d9-a7a3-f9bc60c8cd44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:15.369849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b9249-8ad8-40d9-a7a3-f9bc60c8cd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:15.370117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b9249-8ad8-40d9-a7a3-f9bc60c8cd44 admin admin] PATH_INFO: `/v3/projects/6e7b1c76d7fe4c058e71e0491f57afd9` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:15.371237 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b9249-8ad8-40d9-a7a3-f9bc60c8cd44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e7b1c76d7fe4c058e71e0491f57afd9)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:15.373181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b9249-8ad8-40d9-a7a3-f9bc60c8cd44 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:15.419808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a33b9249-8ad8-40d9-a7a3-f9bc60c8cd44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e7b1c76d7fe4c058e71e0491f57afd9', 'request_id': 'req-a33b9249-8ad8-40d9-a7a3-f9bc60c8cd44'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:15.425208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a33b9249-8ad8-40d9-a7a3-f9bc60c8cd44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e7b1c76d7fe4c058e71e0491f57afd9', 'request_id': 'req-a33b9249-8ad8-40d9-a7a3-f9bc60c8cd44'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:15.431841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a33b9249-8ad8-40d9-a7a3-f9bc60c8cd44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e7b1c76d7fe4c058e71e0491f57afd9', 'request_id': 'req-a33b9249-8ad8-40d9-a7a3-f9bc60c8cd44'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:15.446377 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a33b9249-8ad8-40d9-a7a3-f9bc60c8cd44 admin admin] The token cache is being invalidate because project 6e7b1c76d7fe4c058e71e0491f57afd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:15.447049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a33b9249-8ad8-40d9-a7a3-f9bc60c8cd44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:15.447434 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a33b9249-8ad8-40d9-a7a3-f9bc60c8cd44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:15.448929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1021/2040] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:15 2022] DELETE /identity/v3/projects/6e7b1c76d7fe4c058e71e0491f57afd9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:15.453696 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f6abe-18ae-4c08-8d91-8d2ded07c2dd None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:15.453982 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f6abe-18ae-4c08-8d91-8d2ded07c2dd None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:15.454253 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f6abe-18ae-4c08-8d91-8d2ded07c2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:15.473799 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-740f6abe-18ae-4c08-8d91-8d2ded07c2dd None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:15.475057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-740f6abe-18ae-4c08-8d91-8d2ded07c2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:15.528674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:15.546993 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1020/2041] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:15.555219 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af5500-2306-403f-9232-225d3462ea13 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:15.555494 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af5500-2306-403f-9232-225d3462ea13 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:15.555759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af5500-2306-403f-9232-225d3462ea13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:15.576371 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-28af5500-2306-403f-9232-225d3462ea13 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:15.577571 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-28af5500-2306-403f-9232-225d3462ea13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:15.585869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1022/2042] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:15.590071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f6abe-18ae-4c08-8d91-8d2ded07c2dd None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:15.599638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f6abe-18ae-4c08-8d91-8d2ded07c2dd None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:15.604977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8936049-462c-45de-9f57-1b214c39d45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:15.605596 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8936049-462c-45de-9f57-1b214c39d45e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:15.605872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8936049-462c-45de-9f57-1b214c39d45e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:15.606143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8936049-462c-45de-9f57-1b214c39d45e admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:15.606785 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8936049-462c-45de-9f57-1b214c39d45e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:56:15.607089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8936049-462c-45de-9f57-1b214c39d45e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:15.608773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8936049-462c-45de-9f57-1b214c39d45e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:15.624140 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1021/2043] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:56:15.626162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af5500-2306-403f-9232-225d3462ea13 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:15.637177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af5500-2306-403f-9232-225d3462ea13 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:15.641922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db2bd3-fef0-4694-8bee-a9b2f86d23c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:15.642550 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db2bd3-fef0-4694-8bee-a9b2f86d23c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:15.642825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db2bd3-fef0-4694-8bee-a9b2f86d23c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:15.643095 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db2bd3-fef0-4694-8bee-a9b2f86d23c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:15.643731 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db2bd3-fef0-4694-8bee-a9b2f86d23c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:15.645664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db2bd3-fef0-4694-8bee-a9b2f86d23c7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:15.676117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1023/2044] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:15 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:15.679369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8936049-462c-45de-9f57-1b214c39d45e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:15.689332 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8936049-462c-45de-9f57-1b214c39d45e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:15.745260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b709e55-d8a0-41b7-85ff-559cc5f1bc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:15.745852 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b709e55-d8a0-41b7-85ff-559cc5f1bc8e admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:15.746129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b709e55-d8a0-41b7-85ff-559cc5f1bc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:15.746400 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b709e55-d8a0-41b7-85ff-559cc5f1bc8e admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:15.747028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b709e55-d8a0-41b7-85ff-559cc5f1bc8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:15.748969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b709e55-d8a0-41b7-85ff-559cc5f1bc8e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:15.773200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1022/2045] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:15 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:15.776649 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db2bd3-fef0-4694-8bee-a9b2f86d23c7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:15.787158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db2bd3-fef0-4694-8bee-a9b2f86d23c7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:15.793153 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319d9fc-556c-4e47-adf2-421a9c529a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:15.793862 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319d9fc-556c-4e47-adf2-421a9c529a9e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:15.794202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319d9fc-556c-4e47-adf2-421a9c529a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:15.794560 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319d9fc-556c-4e47-adf2-421a9c529a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:15.795286 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319d9fc-556c-4e47-adf2-421a9c529a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:15.796908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319d9fc-556c-4e47-adf2-421a9c529a9e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:15.808042 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1024/2046] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:15.812029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b709e55-d8a0-41b7-85ff-559cc5f1bc8e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:15.823055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b709e55-d8a0-41b7-85ff-559cc5f1bc8e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:15.827716 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d443473-5c5f-4bde-b918-349449766384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:15.828376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d443473-5c5f-4bde-b918-349449766384 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:15.828656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d443473-5c5f-4bde-b918-349449766384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:15.828926 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d443473-5c5f-4bde-b918-349449766384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:15.829572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d443473-5c5f-4bde-b918-349449766384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:15.831221 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d443473-5c5f-4bde-b918-349449766384 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:15.841948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1023/2047] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:15.846562 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319d9fc-556c-4e47-adf2-421a9c529a9e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:15.857464 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319d9fc-556c-4e47-adf2-421a9c529a9e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:15.862216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9046e-2c95-4337-9319-2a53f13e413b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:15.863871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9046e-2c95-4337-9319-2a53f13e413b admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:15.863871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9046e-2c95-4337-9319-2a53f13e413b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:15.863871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9046e-2c95-4337-9319-2a53f13e413b admin admin] PATH_INFO: `/v3/projects/db936f4f4cea4a94ab81981f9c67c43a/users/30c00fa7ccff436b880ca9d33610141d/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:15.885784 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9046e-2c95-4337-9319-2a53f13e413b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db936f4f4cea4a94ab81981f9c67c43a, user_id=30c00fa7ccff436b880ca9d33610141d, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:15.888646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9046e-2c95-4337-9319-2a53f13e413b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:15.902974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1025/2048] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:15 2022] PUT /identity/v3/projects/db936f4f4cea4a94ab81981f9c67c43a/users/30c00fa7ccff436b880ca9d33610141d/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:56:15.913206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4f440-db93-47d0-b530-0c3045a4d95b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:15.913495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4f440-db93-47d0-b530-0c3045a4d95b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:15.913771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4f440-db93-47d0-b530-0c3045a4d95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:15.946166 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-0eb4f440-db93-47d0-b530-0c3045a4d95b None None] MFA Rules not processed for user `30c00fa7ccff436b880ca9d33610141d`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:15.947416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0eb4f440-db93-47d0-b530-0c3045a4d95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:16.013243 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1024/2049] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:16.018294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9046e-2c95-4337-9319-2a53f13e413b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:16.079389 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9046e-2c95-4337-9319-2a53f13e413b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:16.084561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4f8e2-ccb7-4d92-a8d0-4c5e9ade3a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:16.085296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4f8e2-ccb7-4d92-a8d0-4c5e9ade3a40 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:16.085477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4f8e2-ccb7-4d92-a8d0-4c5e9ade3a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:16.085749 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4f8e2-ccb7-4d92-a8d0-4c5e9ade3a40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:16.086656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4f8e2-ccb7-4d92-a8d0-4c5e9ade3a40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:16.088462 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4f8e2-ccb7-4d92-a8d0-4c5e9ade3a40 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:16.117477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:16.127743 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1026/2050] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:16 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:16.129010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4f440-db93-47d0-b530-0c3045a4d95b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:16.140183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4f440-db93-47d0-b530-0c3045a4d95b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:16.180875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:16.203147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab2d69-2ce6-426d-aeea-87bf98dd8c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:16.203755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab2d69-2ce6-426d-aeea-87bf98dd8c9e admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:16.204032 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab2d69-2ce6-426d-aeea-87bf98dd8c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:16.204332 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab2d69-2ce6-426d-aeea-87bf98dd8c9e admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:16.204955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab2d69-2ce6-426d-aeea-87bf98dd8c9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:16.206871 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab2d69-2ce6-426d-aeea-87bf98dd8c9e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:16.233830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1025/2051] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:16 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:16.237825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4f8e2-ccb7-4d92-a8d0-4c5e9ade3a40 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:16.250029 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4f8e2-ccb7-4d92-a8d0-4c5e9ade3a40 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:16.254680 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5da8ce-b49f-4ad2-bad7-7714cb8d9448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:16.255271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5da8ce-b49f-4ad2-bad7-7714cb8d9448 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:16.255547 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5da8ce-b49f-4ad2-bad7-7714cb8d9448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:16.255816 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5da8ce-b49f-4ad2-bad7-7714cb8d9448 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:16.256480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5da8ce-b49f-4ad2-bad7-7714cb8d9448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:16.258112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5da8ce-b49f-4ad2-bad7-7714cb8d9448 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:16.268532 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1027/2052] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:16.272443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab2d69-2ce6-426d-aeea-87bf98dd8c9e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:16.282222 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab2d69-2ce6-426d-aeea-87bf98dd8c9e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:16.287017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d416e8-7871-4090-b06e-442580669e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:16.287663 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d416e8-7871-4090-b06e-442580669e3d admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:16.288644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d416e8-7871-4090-b06e-442580669e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:16.288981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d416e8-7871-4090-b06e-442580669e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:16.289763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d416e8-7871-4090-b06e-442580669e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:16.291351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d416e8-7871-4090-b06e-442580669e3d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:16.301306 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1026/2053] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:16.306241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5da8ce-b49f-4ad2-bad7-7714cb8d9448 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:16.315980 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5da8ce-b49f-4ad2-bad7-7714cb8d9448 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:16.321098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976618c-3c7b-4c3e-80a4-8de2674d9528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:16.321728 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976618c-3c7b-4c3e-80a4-8de2674d9528 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:16.322003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976618c-3c7b-4c3e-80a4-8de2674d9528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:16.322273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976618c-3c7b-4c3e-80a4-8de2674d9528 admin admin] PATH_INFO: `/v3/projects/d1346c8ba4b84f84a1f333c4452157fe/users/7cf7e144183d41d194bb61e55b3c518e/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:16.336424 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976618c-3c7b-4c3e-80a4-8de2674d9528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1346c8ba4b84f84a1f333c4452157fe, user_id=7cf7e144183d41d194bb61e55b3c518e, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:16.339012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976618c-3c7b-4c3e-80a4-8de2674d9528 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:16.352802 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1028/2054] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:16 2022] PUT /identity/v3/projects/d1346c8ba4b84f84a1f333c4452157fe/users/7cf7e144183d41d194bb61e55b3c518e/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:56:16.364374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bab284-23b0-425a-9568-5d43e97d8eac None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:16.364820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bab284-23b0-425a-9568-5d43e97d8eac None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:16.365188 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bab284-23b0-425a-9568-5d43e97d8eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:16.401116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-80bab284-23b0-425a-9568-5d43e97d8eac None None] MFA Rules not processed for user `7cf7e144183d41d194bb61e55b3c518e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:16.402448 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-80bab284-23b0-425a-9568-5d43e97d8eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:16.478592 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1027/2055] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:16.482868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976618c-3c7b-4c3e-80a4-8de2674d9528 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:16.557802 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976618c-3c7b-4c3e-80a4-8de2674d9528 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:16.562419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570ca05-f123-413c-9e45-635efbe8c00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:16.563010 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570ca05-f123-413c-9e45-635efbe8c00b admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:16.563286 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570ca05-f123-413c-9e45-635efbe8c00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:16.563551 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570ca05-f123-413c-9e45-635efbe8c00b admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:16.564208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570ca05-f123-413c-9e45-635efbe8c00b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:16.566100 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570ca05-f123-413c-9e45-635efbe8c00b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:16.593002 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1029/2056] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:16 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:16.597355 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bab284-23b0-425a-9568-5d43e97d8eac None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:16.608831 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bab284-23b0-425a-9568-5d43e97d8eac None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:16.666874 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91825f13-2858-48c7-973e-f4886cdceda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:16.667487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91825f13-2858-48c7-973e-f4886cdceda8 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:16.667765 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91825f13-2858-48c7-973e-f4886cdceda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:16.668035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91825f13-2858-48c7-973e-f4886cdceda8 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:16.668684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91825f13-2858-48c7-973e-f4886cdceda8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:16.670583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91825f13-2858-48c7-973e-f4886cdceda8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:16.697332 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1028/2057] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:16 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:16.700941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570ca05-f123-413c-9e45-635efbe8c00b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:16.716094 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570ca05-f123-413c-9e45-635efbe8c00b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:16.723443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72ea22-f6b5-49e4-b6ef-a6f5f76d4e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:16.724400 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72ea22-f6b5-49e4-b6ef-a6f5f76d4e55 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:16.725255 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72ea22-f6b5-49e4-b6ef-a6f5f76d4e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:16.725368 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72ea22-f6b5-49e4-b6ef-a6f5f76d4e55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:16.726713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da72ea22-f6b5-49e4-b6ef-a6f5f76d4e55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:16.729418 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da72ea22-f6b5-49e4-b6ef-a6f5f76d4e55 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:16.748126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1030/2058] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:16.752129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91825f13-2858-48c7-973e-f4886cdceda8 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:16.763155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91825f13-2858-48c7-973e-f4886cdceda8 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:16.768962 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e62914-913e-43c2-a96c-fd0d87620411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:16.769747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e62914-913e-43c2-a96c-fd0d87620411 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:16.770096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e62914-913e-43c2-a96c-fd0d87620411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:16.770439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e62914-913e-43c2-a96c-fd0d87620411 admin admin] PATH_INFO: `/v3/projects/d4f1d90ef7fa4606890bea7b35c94538/users/47335afeca6044e298509414f5cc1910/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:16.789146 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e62914-913e-43c2-a96c-fd0d87620411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4f1d90ef7fa4606890bea7b35c94538, user_id=47335afeca6044e298509414f5cc1910, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:16.791965 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e62914-913e-43c2-a96c-fd0d87620411 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:16.812171 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1029/2059] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:16 2022] PUT /identity/v3/projects/d4f1d90ef7fa4606890bea7b35c94538/users/47335afeca6044e298509414f5cc1910/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:16.816521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72ea22-f6b5-49e4-b6ef-a6f5f76d4e55 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:16.828877 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72ea22-f6b5-49e4-b6ef-a6f5f76d4e55 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:16.899780 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d7d725-03a8-4ec9-b76b-3580adabe351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:16.900540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7d725-03a8-4ec9-b76b-3580adabe351 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:16.900928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7d725-03a8-4ec9-b76b-3580adabe351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:16.901235 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7d725-03a8-4ec9-b76b-3580adabe351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:16.901892 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d7d725-03a8-4ec9-b76b-3580adabe351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:16.903728 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d7d725-03a8-4ec9-b76b-3580adabe351 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:16.915717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1031/2060] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:16.920239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e62914-913e-43c2-a96c-fd0d87620411 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:16.931058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e62914-913e-43c2-a96c-fd0d87620411 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:16.936755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccabae1-1230-409c-9aa0-d5d900ce9c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:16.937524 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccabae1-1230-409c-9aa0-d5d900ce9c7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:16.937800 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccabae1-1230-409c-9aa0-d5d900ce9c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:16.938074 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccabae1-1230-409c-9aa0-d5d900ce9c7b admin admin] PATH_INFO: `/v3/projects/d4f1d90ef7fa4606890bea7b35c94538/users/47335afeca6044e298509414f5cc1910/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:16.940028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccabae1-1230-409c-9aa0-d5d900ce9c7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4f1d90ef7fa4606890bea7b35c94538, user_id=47335afeca6044e298509414f5cc1910, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:16.943028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccabae1-1230-409c-9aa0-d5d900ce9c7b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:16.955306 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1030/2061] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:16 2022] PUT /identity/v3/projects/d4f1d90ef7fa4606890bea7b35c94538/users/47335afeca6044e298509414f5cc1910/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:16.966222 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175a92-5fa4-462b-b4d4-c6196e6d0165 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:16.966605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175a92-5fa4-462b-b4d4-c6196e6d0165 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:16.966950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175a92-5fa4-462b-b4d4-c6196e6d0165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:16.997531 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9c175a92-5fa4-462b-b4d4-c6196e6d0165 None None] MFA Rules not processed for user `47335afeca6044e298509414f5cc1910`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:16.998874 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9c175a92-5fa4-462b-b4d4-c6196e6d0165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:17.030807 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:17.087958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1032/2062] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:17.099306 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccabae1-1230-409c-9aa0-d5d900ce9c7b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:17.101146 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6ccabae1-1230-409c-9aa0-d5d900ce9c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:17.108547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:17.182447 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccabae1-1230-409c-9aa0-d5d900ce9c7b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:17.190046 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2190173-2535-44f5-ad17-8b415da71598 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:17.190769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2190173-2535-44f5-ad17-8b415da71598 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:17.191085 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2190173-2535-44f5-ad17-8b415da71598 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:17.191765 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2190173-2535-44f5-ad17-8b415da71598 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:17.211397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2190173-2535-44f5-ad17-8b415da71598 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:17.213523 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2190173-2535-44f5-ad17-8b415da71598 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:17.227508 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1031/2063] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:17.864557 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c175a92-5fa4-462b-b4d4-c6196e6d0165 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:17.866253 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9c175a92-5fa4-462b-b4d4-c6196e6d0165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:17.879263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c175a92-5fa4-462b-b4d4-c6196e6d0165 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:17.897051 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f439b0-c353-4236-b468-2d24fefe3290 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:17.899531 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f439b0-c353-4236-b468-2d24fefe3290 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:17.899634 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f439b0-c353-4236-b468-2d24fefe3290 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:17.900062 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f439b0-c353-4236-b468-2d24fefe3290 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:17.996732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f439b0-c353-4236-b468-2d24fefe3290 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:17.998466 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f439b0-c353-4236-b468-2d24fefe3290 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:18.010089 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1033/2064] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:19.019518 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2190173-2535-44f5-ad17-8b415da71598 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:19.021048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f2190173-2535-44f5-ad17-8b415da71598 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:19.086002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2190173-2535-44f5-ad17-8b415da71598 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:19.090459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e507ad-6a70-4c8b-8133-b36ff783a1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:19.091085 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e507ad-6a70-4c8b-8133-b36ff783a1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:19.091358 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e507ad-6a70-4c8b-8133-b36ff783a1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:19.091627 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e507ad-6a70-4c8b-8133-b36ff783a1bf admin admin] PATH_INFO: `/v3/users/694a1a7671c540aeb3fddba18cf67e13` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:19.092751 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e507ad-6a70-4c8b-8133-b36ff783a1bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=694a1a7671c540aeb3fddba18cf67e13)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:19.094734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e507ad-6a70-4c8b-8133-b36ff783a1bf admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:19.141529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-34e507ad-6a70-4c8b-8133-b36ff783a1bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '694a1a7671c540aeb3fddba18cf67e13'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:19.142188 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-34e507ad-6a70-4c8b-8133-b36ff783a1bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '694a1a7671c540aeb3fddba18cf67e13'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:19.154416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-34e507ad-6a70-4c8b-8133-b36ff783a1bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '694a1a7671c540aeb3fddba18cf67e13'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:19.155064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-34e507ad-6a70-4c8b-8133-b36ff783a1bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '694a1a7671c540aeb3fddba18cf67e13'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:19.162405 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-34e507ad-6a70-4c8b-8133-b36ff783a1bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '694a1a7671c540aeb3fddba18cf67e13'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:19.174400 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1032/2065] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:19 2022] DELETE /identity/v3/users/694a1a7671c540aeb3fddba18cf67e13 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:19.179436 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ff306-3909-4183-95a6-58fa65ab00b7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:19.179911 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ff306-3909-4183-95a6-58fa65ab00b7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:19.180001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ff306-3909-4183-95a6-58fa65ab00b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:19.199585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-233ff306-3909-4183-95a6-58fa65ab00b7 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:19.200817 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-233ff306-3909-4183-95a6-58fa65ab00b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:19.261691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1034/2066] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:56:19.271839 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e507ad-6a70-4c8b-8133-b36ff783a1bf admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:19.274235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-34e507ad-6a70-4c8b-8133-b36ff783a1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:19.308530 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:19.341440 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e507ad-6a70-4c8b-8133-b36ff783a1bf admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:19.345272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84325bda-dfdb-40e1-8596-7a6d1d7f505a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:19.345896 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84325bda-dfdb-40e1-8596-7a6d1d7f505a service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:19.346197 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84325bda-dfdb-40e1-8596-7a6d1d7f505a service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:19.346463 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84325bda-dfdb-40e1-8596-7a6d1d7f505a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:19.356955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84325bda-dfdb-40e1-8596-7a6d1d7f505a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:19.358367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84325bda-dfdb-40e1-8596-7a6d1d7f505a service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:19.368412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1033/2067] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:19.671457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ff306-3909-4183-95a6-58fa65ab00b7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:19.672937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-233ff306-3909-4183-95a6-58fa65ab00b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:19.685441 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ff306-3909-4183-95a6-58fa65ab00b7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:19.690108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656e52e-fc0c-4514-9db6-32d6037492e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:19.690726 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656e52e-fc0c-4514-9db6-32d6037492e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:19.690999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656e52e-fc0c-4514-9db6-32d6037492e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:19.691517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656e52e-fc0c-4514-9db6-32d6037492e9 admin admin] PATH_INFO: `/v3/projects/686f07a065834096a008f8839f220d15` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:19.692660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656e52e-fc0c-4514-9db6-32d6037492e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=686f07a065834096a008f8839f220d15)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:19.695182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656e52e-fc0c-4514-9db6-32d6037492e9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:19.758671 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b656e52e-fc0c-4514-9db6-32d6037492e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '686f07a065834096a008f8839f220d15', 'request_id': 'req-b656e52e-fc0c-4514-9db6-32d6037492e9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:19.763412 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:19.766606 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b656e52e-fc0c-4514-9db6-32d6037492e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '686f07a065834096a008f8839f220d15', 'request_id': 'req-b656e52e-fc0c-4514-9db6-32d6037492e9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:19.776164 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b656e52e-fc0c-4514-9db6-32d6037492e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '686f07a065834096a008f8839f220d15', 'request_id': 'req-b656e52e-fc0c-4514-9db6-32d6037492e9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:19.793858 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b656e52e-fc0c-4514-9db6-32d6037492e9 admin admin] The token cache is being invalidate because project 686f07a065834096a008f8839f220d15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:19.794156 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b656e52e-fc0c-4514-9db6-32d6037492e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:19.794834 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b656e52e-fc0c-4514-9db6-32d6037492e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:19.799927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1035/2068] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:19 2022] DELETE /identity/v3/projects/686f07a065834096a008f8839f220d15 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:19.801705 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bce7a-5a7b-476a-97a3-7063b80ca689 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:19.802063 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bce7a-5a7b-476a-97a3-7063b80ca689 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:19.802963 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bce7a-5a7b-476a-97a3-7063b80ca689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:19.828812 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-6a6bce7a-5a7b-476a-97a3-7063b80ca689 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:19.830410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6a6bce7a-5a7b-476a-97a3-7063b80ca689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:19.908720 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1034/2069] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:19.910967 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b2c0f-63dc-400a-8eaf-8a2b55c8d215 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:19.911567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b2c0f-63dc-400a-8eaf-8a2b55c8d215 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:19.912290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b2c0f-63dc-400a-8eaf-8a2b55c8d215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:19.955339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-bc0b2c0f-63dc-400a-8eaf-8a2b55c8d215 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:19.956883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bc0b2c0f-63dc-400a-8eaf-8a2b55c8d215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:19.979414 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1036/2070] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:19.983689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bce7a-5a7b-476a-97a3-7063b80ca689 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:19.995677 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bce7a-5a7b-476a-97a3-7063b80ca689 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:20.020944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c59499-7e6d-420a-8194-592914079cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:20.025373 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c59499-7e6d-420a-8194-592914079cdb admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:20.025916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c59499-7e6d-420a-8194-592914079cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:20.028495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c59499-7e6d-420a-8194-592914079cdb admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:20.029348 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c59499-7e6d-420a-8194-592914079cdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:56:20.029793 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c59499-7e6d-420a-8194-592914079cdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:20.031931 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c59499-7e6d-420a-8194-592914079cdb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:20.060282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1035/2071] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:56:20.065970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b2c0f-63dc-400a-8eaf-8a2b55c8d215 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:20.080054 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b2c0f-63dc-400a-8eaf-8a2b55c8d215 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:20.087457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0712c-33ef-4118-8b99-13f015700529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:20.088635 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0712c-33ef-4118-8b99-13f015700529 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:20.088931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0712c-33ef-4118-8b99-13f015700529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:20.089198 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0712c-33ef-4118-8b99-13f015700529 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:20.090445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0712c-33ef-4118-8b99-13f015700529 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:20.097361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0712c-33ef-4118-8b99-13f015700529 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:20.147566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1037/2072] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:20 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:56:20.148986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c59499-7e6d-420a-8194-592914079cdb admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:20.174683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c59499-7e6d-420a-8194-592914079cdb admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:20.262617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ec48e-efa0-44a4-bc32-c3deecb29315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:20.263231 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ec48e-efa0-44a4-bc32-c3deecb29315 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:20.263510 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ec48e-efa0-44a4-bc32-c3deecb29315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:20.263784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ec48e-efa0-44a4-bc32-c3deecb29315 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:20.264473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ec48e-efa0-44a4-bc32-c3deecb29315 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:20.266580 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ec48e-efa0-44a4-bc32-c3deecb29315 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:20.291998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1036/2073] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:20 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:56:20.296022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0712c-33ef-4118-8b99-13f015700529 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:20.308333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0712c-33ef-4118-8b99-13f015700529 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:20.324356 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd8f64-1980-4f98-8278-01930f2aa34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:20.327112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd8f64-1980-4f98-8278-01930f2aa34b admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:20.327549 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd8f64-1980-4f98-8278-01930f2aa34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:20.328463 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd8f64-1980-4f98-8278-01930f2aa34b admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:20.329308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd8f64-1980-4f98-8278-01930f2aa34b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:20.332046 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd8f64-1980-4f98-8278-01930f2aa34b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:20.345345 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1038/2074] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:20.349335 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ec48e-efa0-44a4-bc32-c3deecb29315 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:20.368554 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ec48e-efa0-44a4-bc32-c3deecb29315 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:20.375740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27fb85-a350-4d25-9da9-e16833ef5360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:20.375740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27fb85-a350-4d25-9da9-e16833ef5360 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:20.375740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27fb85-a350-4d25-9da9-e16833ef5360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:20.375740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27fb85-a350-4d25-9da9-e16833ef5360 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:20.376107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c27fb85-a350-4d25-9da9-e16833ef5360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:20.376393 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c27fb85-a350-4d25-9da9-e16833ef5360 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:20.406094 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd8f64-1980-4f98-8278-01930f2aa34b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:20.406708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1037/2075] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:20.420309 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd8f64-1980-4f98-8278-01930f2aa34b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:20.425506 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d1068-a175-4543-b789-898ba0a4a812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:20.426171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d1068-a175-4543-b789-898ba0a4a812 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:20.426626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d1068-a175-4543-b789-898ba0a4a812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:20.426905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d1068-a175-4543-b789-898ba0a4a812 admin admin] PATH_INFO: `/v3/projects/126a90736fb3438ab05ffeae4bafa7a1/users/f018f290f36f4121becdfcc2bc10827c/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:20.443826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d1068-a175-4543-b789-898ba0a4a812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=126a90736fb3438ab05ffeae4bafa7a1, user_id=f018f290f36f4121becdfcc2bc10827c, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:20.446968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d1068-a175-4543-b789-898ba0a4a812 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:20.459522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1039/2076] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:20 2022] PUT /identity/v3/projects/126a90736fb3438ab05ffeae4bafa7a1/users/f018f290f36f4121becdfcc2bc10827c/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:20.465124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce7df0-6a0b-4e55-ae5e-700d225d7800 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:20.465408 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce7df0-6a0b-4e55-ae5e-700d225d7800 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:20.465679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce7df0-6a0b-4e55-ae5e-700d225d7800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:20.486343 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-70ce7df0-6a0b-4e55-ae5e-700d225d7800 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:20.487547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-70ce7df0-6a0b-4e55-ae5e-700d225d7800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:20.515553 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:20.545781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1038/2077] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:20.555243 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d1068-a175-4543-b789-898ba0a4a812 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:20.556935 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-163d1068-a175-4543-b789-898ba0a4a812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:20.615002 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d1068-a175-4543-b789-898ba0a4a812 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:20.618591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652e795-79d7-4ad7-b18a-6fa999d2461a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:20.619178 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652e795-79d7-4ad7-b18a-6fa999d2461a service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:20.619453 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652e795-79d7-4ad7-b18a-6fa999d2461a service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:20.619722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652e795-79d7-4ad7-b18a-6fa999d2461a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:20.628454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652e795-79d7-4ad7-b18a-6fa999d2461a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:20.629857 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652e795-79d7-4ad7-b18a-6fa999d2461a service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:20.639197 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1040/2078] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:24.759310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce7df0-6a0b-4e55-ae5e-700d225d7800 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:24.760998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-70ce7df0-6a0b-4e55-ae5e-700d225d7800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:24.836015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce7df0-6a0b-4e55-ae5e-700d225d7800 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:24.840466 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899fca4-fcfb-4513-8b22-143c3c652868 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:24.841097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899fca4-fcfb-4513-8b22-143c3c652868 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:24.841468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899fca4-fcfb-4513-8b22-143c3c652868 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:24.841827 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899fca4-fcfb-4513-8b22-143c3c652868 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:24.843670 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f899fca4-fcfb-4513-8b22-143c3c652868 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:25.008382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899fca4-fcfb-4513-8b22-143c3c652868 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:25.019072 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899fca4-fcfb-4513-8b22-143c3c652868 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:25.080058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1039/2079] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:24 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:26.460828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902db89-ff47-4eb5-b8d2-1bf9c91f3914 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:26.461987 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902db89-ff47-4eb5-b8d2-1bf9c91f3914 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:26.462471 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902db89-ff47-4eb5-b8d2-1bf9c91f3914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:26.501672 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a902db89-ff47-4eb5-b8d2-1bf9c91f3914 None None] MFA Rules not processed for user `f018f290f36f4121becdfcc2bc10827c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:26.503366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a902db89-ff47-4eb5-b8d2-1bf9c91f3914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:26.602096 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1041/2080] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:26.606687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899fca4-fcfb-4513-8b22-143c3c652868 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:26.618732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899fca4-fcfb-4513-8b22-143c3c652868 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:26.624558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d0711-6f6b-4a18-9eb6-fd13f27dbf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:26.625481 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d0711-6f6b-4a18-9eb6-fd13f27dbf4b admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:26.625976 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d0711-6f6b-4a18-9eb6-fd13f27dbf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:26.626442 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d0711-6f6b-4a18-9eb6-fd13f27dbf4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:26.627533 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d0711-6f6b-4a18-9eb6-fd13f27dbf4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:26.629686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d0711-6f6b-4a18-9eb6-fd13f27dbf4b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:26.675732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1040/2081] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:26 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:26.676491 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902db89-ff47-4eb5-b8d2-1bf9c91f3914 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:26.696406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902db89-ff47-4eb5-b8d2-1bf9c91f3914 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:26.713557 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:26.805277 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed79031-14b5-494a-8534-cd893c364bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:26.806034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed79031-14b5-494a-8534-cd893c364bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:26.806390 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed79031-14b5-494a-8534-cd893c364bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:26.806735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed79031-14b5-494a-8534-cd893c364bc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:26.807478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed79031-14b5-494a-8534-cd893c364bc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:26.809597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed79031-14b5-494a-8534-cd893c364bc0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:26.842561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d0711-6f6b-4a18-9eb6-fd13f27dbf4b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:26.843533 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1042/2082] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:26 2022] POST /identity/v3/users => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:56:26.855771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d0711-6f6b-4a18-9eb6-fd13f27dbf4b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:26.860534 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084c63d-d2f8-49cf-8f8f-32eeb95a7436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:26.861283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084c63d-d2f8-49cf-8f8f-32eeb95a7436 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:26.861561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084c63d-d2f8-49cf-8f8f-32eeb95a7436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:26.861827 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084c63d-d2f8-49cf-8f8f-32eeb95a7436 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:26.862464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084c63d-d2f8-49cf-8f8f-32eeb95a7436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:26.864272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084c63d-d2f8-49cf-8f8f-32eeb95a7436 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:26.880699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1041/2083] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:26.881869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed79031-14b5-494a-8534-cd893c364bc0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:26.894559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed79031-14b5-494a-8534-cd893c364bc0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:26.906341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc7ce5-70c1-4271-937a-5951d5f66e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:26.908443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc7ce5-70c1-4271-937a-5951d5f66e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:26.908811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc7ce5-70c1-4271-937a-5951d5f66e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:26.909169 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc7ce5-70c1-4271-937a-5951d5f66e8e admin admin] PATH_INFO: `/v3/projects/6574ad7065204f13a74970802ca48371/users/935523b362544defa4596ebff6c8aa82/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:26.930789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc7ce5-70c1-4271-937a-5951d5f66e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6574ad7065204f13a74970802ca48371, user_id=935523b362544defa4596ebff6c8aa82, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:26.933805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc7ce5-70c1-4271-937a-5951d5f66e8e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:26.961641 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084c63d-d2f8-49cf-8f8f-32eeb95a7436 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:26.964158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1043/2084] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:26 2022] PUT /identity/v3/projects/6574ad7065204f13a74970802ca48371/users/935523b362544defa4596ebff6c8aa82/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:56:26.972142 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084c63d-d2f8-49cf-8f8f-32eeb95a7436 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:27.073661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:27.082241 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261ce15-e3d5-4e9b-bee3-17ae7b8bb874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:27.083262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261ce15-e3d5-4e9b-bee3-17ae7b8bb874 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:27.085425 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261ce15-e3d5-4e9b-bee3-17ae7b8bb874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:27.086121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261ce15-e3d5-4e9b-bee3-17ae7b8bb874 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:27.086833 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5261ce15-e3d5-4e9b-bee3-17ae7b8bb874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:27.088628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5261ce15-e3d5-4e9b-bee3-17ae7b8bb874 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:27.104171 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1042/2085] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:27.105999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc7ce5-70c1-4271-937a-5951d5f66e8e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:27.117714 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc7ce5-70c1-4271-937a-5951d5f66e8e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:27.122738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c1146-5841-4d5a-90cd-3285488c5302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:27.123417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c1146-5841-4d5a-90cd-3285488c5302 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:27.123698 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c1146-5841-4d5a-90cd-3285488c5302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:27.123973 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c1146-5841-4d5a-90cd-3285488c5302 admin admin] PATH_INFO: `/v3/projects/6574ad7065204f13a74970802ca48371/users/935523b362544defa4596ebff6c8aa82/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:27.126156 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c1146-5841-4d5a-90cd-3285488c5302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6574ad7065204f13a74970802ca48371, user_id=935523b362544defa4596ebff6c8aa82, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:27.129342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c1146-5841-4d5a-90cd-3285488c5302 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:27.148802 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1044/2086] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:27 2022] PUT /identity/v3/projects/6574ad7065204f13a74970802ca48371/users/935523b362544defa4596ebff6c8aa82/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:32.123025 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261ce15-e3d5-4e9b-bee3-17ae7b8bb874 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:32.124825 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5261ce15-e3d5-4e9b-bee3-17ae7b8bb874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:32.151424 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261ce15-e3d5-4e9b-bee3-17ae7b8bb874 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:32.282967 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542e803-79f6-4261-8248-2eaa0616a799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:32.283612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542e803-79f6-4261-8248-2eaa0616a799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:32.283876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542e803-79f6-4261-8248-2eaa0616a799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:32.284167 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542e803-79f6-4261-8248-2eaa0616a799 admin admin] PATH_INFO: `/v3/users/30c00fa7ccff436b880ca9d33610141d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:32.298473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542e803-79f6-4261-8248-2eaa0616a799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30c00fa7ccff436b880ca9d33610141d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:32.300754 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542e803-79f6-4261-8248-2eaa0616a799 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:32.405087 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c542e803-79f6-4261-8248-2eaa0616a799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30c00fa7ccff436b880ca9d33610141d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:32.406522 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c542e803-79f6-4261-8248-2eaa0616a799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30c00fa7ccff436b880ca9d33610141d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:32.424462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c542e803-79f6-4261-8248-2eaa0616a799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30c00fa7ccff436b880ca9d33610141d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:32.429352 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c542e803-79f6-4261-8248-2eaa0616a799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30c00fa7ccff436b880ca9d33610141d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:32.453344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c542e803-79f6-4261-8248-2eaa0616a799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30c00fa7ccff436b880ca9d33610141d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:32.485627 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1043/2087] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:32 2022] DELETE /identity/v3/users/30c00fa7ccff436b880ca9d33610141d => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:32.491467 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34d923-ae03-4c4e-954f-80015cee03b5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:32.491968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34d923-ae03-4c4e-954f-80015cee03b5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:32.492296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34d923-ae03-4c4e-954f-80015cee03b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:32.531246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8b34d923-ae03-4c4e-954f-80015cee03b5 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:32.533157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8b34d923-ae03-4c4e-954f-80015cee03b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:32.644982 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1045/2088] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:32.654131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542e803-79f6-4261-8248-2eaa0616a799 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:32.656197 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c542e803-79f6-4261-8248-2eaa0616a799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:32.758471 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542e803-79f6-4261-8248-2eaa0616a799 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:32.762548 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d6989-638b-48b8-b8ea-cc292dc631e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:32.763250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d6989-638b-48b8-b8ea-cc292dc631e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:32.763590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d6989-638b-48b8-b8ea-cc292dc631e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:32.764812 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d6989-638b-48b8-b8ea-cc292dc631e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:32.775623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d6989-638b-48b8-b8ea-cc292dc631e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:32.777601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d6989-638b-48b8-b8ea-cc292dc631e5 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:32.792235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1044/2089] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:33.319029 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34d923-ae03-4c4e-954f-80015cee03b5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:33.320679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8b34d923-ae03-4c4e-954f-80015cee03b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:33.335294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34d923-ae03-4c4e-954f-80015cee03b5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:33.340192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858d670-60cd-4218-a86c-8f62326e9604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:33.340842 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858d670-60cd-4218-a86c-8f62326e9604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:33.341126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858d670-60cd-4218-a86c-8f62326e9604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:33.341388 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858d670-60cd-4218-a86c-8f62326e9604 admin admin] PATH_INFO: `/v3/projects/db936f4f4cea4a94ab81981f9c67c43a` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:33.342561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858d670-60cd-4218-a86c-8f62326e9604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db936f4f4cea4a94ab81981f9c67c43a)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:33.344561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858d670-60cd-4218-a86c-8f62326e9604 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:33.416363 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9858d670-60cd-4218-a86c-8f62326e9604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db936f4f4cea4a94ab81981f9c67c43a', 'request_id': 'req-9858d670-60cd-4218-a86c-8f62326e9604'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:33.420451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9858d670-60cd-4218-a86c-8f62326e9604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db936f4f4cea4a94ab81981f9c67c43a', 'request_id': 'req-9858d670-60cd-4218-a86c-8f62326e9604'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:33.422605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:33.437261 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9858d670-60cd-4218-a86c-8f62326e9604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db936f4f4cea4a94ab81981f9c67c43a', 'request_id': 'req-9858d670-60cd-4218-a86c-8f62326e9604'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:33.466223 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9858d670-60cd-4218-a86c-8f62326e9604 admin admin] The token cache is being invalidate because project db936f4f4cea4a94ab81981f9c67c43a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:33.466488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9858d670-60cd-4218-a86c-8f62326e9604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:33.467251 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9858d670-60cd-4218-a86c-8f62326e9604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:33.469895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1046/2090] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:33 2022] DELETE /identity/v3/projects/db936f4f4cea4a94ab81981f9c67c43a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:33.473895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d6989-638b-48b8-b8ea-cc292dc631e5 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:33.495606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d6989-638b-48b8-b8ea-cc292dc631e5 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:33.541828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:33.625261 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858d670-60cd-4218-a86c-8f62326e9604 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:33.627695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9858d670-60cd-4218-a86c-8f62326e9604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:33.709289 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78e161-1ce9-4804-a732-c768bfaaaff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:33.710393 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78e161-1ce9-4804-a732-c768bfaaaff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:33.710940 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78e161-1ce9-4804-a732-c768bfaaaff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:33.711412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78e161-1ce9-4804-a732-c768bfaaaff8 admin admin] PATH_INFO: `/v3/users/7cf7e144183d41d194bb61e55b3c518e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:33.714430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78e161-1ce9-4804-a732-c768bfaaaff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf7e144183d41d194bb61e55b3c518e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:33.716296 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78e161-1ce9-4804-a732-c768bfaaaff8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:33.795382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858d670-60cd-4218-a86c-8f62326e9604 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:33.802371 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef389e0b-d3b2-485e-a8bd-bc306ca5dfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:33.803096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7d78e161-1ce9-4804-a732-c768bfaaaff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf7e144183d41d194bb61e55b3c518e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:33.803417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef389e0b-d3b2-485e-a8bd-bc306ca5dfdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:33.803723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef389e0b-d3b2-485e-a8bd-bc306ca5dfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:33.803948 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef389e0b-d3b2-485e-a8bd-bc306ca5dfdc admin admin] PATH_INFO: `/v3/users/ccc30d896f184a12b30fc8705da46f95` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:33.805399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef389e0b-d3b2-485e-a8bd-bc306ca5dfdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccc30d896f184a12b30fc8705da46f95)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:33.805759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7d78e161-1ce9-4804-a732-c768bfaaaff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf7e144183d41d194bb61e55b3c518e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:33.807913 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef389e0b-d3b2-485e-a8bd-bc306ca5dfdc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:33.822331 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7d78e161-1ce9-4804-a732-c768bfaaaff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf7e144183d41d194bb61e55b3c518e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:33.823382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7d78e161-1ce9-4804-a732-c768bfaaaff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf7e144183d41d194bb61e55b3c518e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:33.831989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7d78e161-1ce9-4804-a732-c768bfaaaff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf7e144183d41d194bb61e55b3c518e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:33.863645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1045/2091] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:33 2022] DELETE /identity/v3/users/7cf7e144183d41d194bb61e55b3c518e => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:33.866575 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c893e-bf30-4bd8-a943-bc19fabf9812 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:33.866980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c893e-bf30-4bd8-a943-bc19fabf9812 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:33.867315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c893e-bf30-4bd8-a943-bc19fabf9812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:33.885494 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ef389e0b-d3b2-485e-a8bd-bc306ca5dfdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccc30d896f184a12b30fc8705da46f95'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:33.906110 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-690c893e-bf30-4bd8-a943-bc19fabf9812 None None] MFA Rules not processed for user `935523b362544defa4596ebff6c8aa82`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:33.908115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-690c893e-bf30-4bd8-a943-bc19fabf9812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:33.908983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ef389e0b-d3b2-485e-a8bd-bc306ca5dfdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccc30d896f184a12b30fc8705da46f95'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:33.925703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ef389e0b-d3b2-485e-a8bd-bc306ca5dfdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccc30d896f184a12b30fc8705da46f95'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:33.926341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ef389e0b-d3b2-485e-a8bd-bc306ca5dfdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccc30d896f184a12b30fc8705da46f95'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:33.936344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ef389e0b-d3b2-485e-a8bd-bc306ca5dfdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccc30d896f184a12b30fc8705da46f95'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:33.947499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1047/2092] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:33 2022] DELETE /identity/v3/users/ccc30d896f184a12b30fc8705da46f95 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:33.951827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e527c5-29f7-49ba-9df1-d0523dbf30c9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:33.951827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e527c5-29f7-49ba-9df1-d0523dbf30c9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:33.952187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e527c5-29f7-49ba-9df1-d0523dbf30c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:33.980643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-74e527c5-29f7-49ba-9df1-d0523dbf30c9 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:33.982788 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-74e527c5-29f7-49ba-9df1-d0523dbf30c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:34.010323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1046/2093] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:34.023658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c893e-bf30-4bd8-a943-bc19fabf9812 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:34.025362 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-690c893e-bf30-4bd8-a943-bc19fabf9812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:34.056566 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:34.086465 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1048/2094] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:34.097287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e527c5-29f7-49ba-9df1-d0523dbf30c9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:34.099380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-74e527c5-29f7-49ba-9df1-d0523dbf30c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:34.111513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:34.154417 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c893e-bf30-4bd8-a943-bc19fabf9812 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:34.184396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e527c5-29f7-49ba-9df1-d0523dbf30c9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:34.188300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b6cfe-9b58-493b-b6e1-3ff6ba50c7ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:34.189021 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b6cfe-9b58-493b-b6e1-3ff6ba50c7ab service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:34.189306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b6cfe-9b58-493b-b6e1-3ff6ba50c7ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:34.189574 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b6cfe-9b58-493b-b6e1-3ff6ba50c7ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:34.200017 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b6cfe-9b58-493b-b6e1-3ff6ba50c7ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:34.201467 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b6cfe-9b58-493b-b6e1-3ff6ba50c7ab service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:34.212118 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1049/2095] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 07:56:34.220763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43491b65-a66b-48c8-b4e1-64acfe4faf4d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b0652c89c9945e2866512c9da4793ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:34.221586 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43491b65-a66b-48c8-b4e1-64acfe4faf4d service glance] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:34.221968 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43491b65-a66b-48c8-b4e1-64acfe4faf4d service glance] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:34.222262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43491b65-a66b-48c8-b4e1-64acfe4faf4d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:34.224441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-43491b65-a66b-48c8-b4e1-64acfe4faf4d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:34.231248 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b6cfe-9b58-493b-b6e1-3ff6ba50c7ab service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:34.236395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f90b6cfe-9b58-493b-b6e1-3ff6ba50c7ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:34.261928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b6cfe-9b58-493b-b6e1-3ff6ba50c7ab service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:34.267602 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15094aac-0c9d-4b61-a659-292da6907f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:34.268325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15094aac-0c9d-4b61-a659-292da6907f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:34.268608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15094aac-0c9d-4b61-a659-292da6907f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:34.268892 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15094aac-0c9d-4b61-a659-292da6907f30 admin admin] PATH_INFO: `/v3/projects/d1346c8ba4b84f84a1f333c4452157fe` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:34.270473 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15094aac-0c9d-4b61-a659-292da6907f30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1346c8ba4b84f84a1f333c4452157fe)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:34.273319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15094aac-0c9d-4b61-a659-292da6907f30 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:34.299938 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43491b65-a66b-48c8-b4e1-64acfe4faf4d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:34.301822 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43491b65-a66b-48c8-b4e1-64acfe4faf4d service glance] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:34.315483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1047/2096] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 07:56:34.346241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-15094aac-0c9d-4b61-a659-292da6907f30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1346c8ba4b84f84a1f333c4452157fe', 'request_id': 'req-15094aac-0c9d-4b61-a659-292da6907f30'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:34.355089 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-15094aac-0c9d-4b61-a659-292da6907f30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1346c8ba4b84f84a1f333c4452157fe', 'request_id': 'req-15094aac-0c9d-4b61-a659-292da6907f30'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:34.363948 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-15094aac-0c9d-4b61-a659-292da6907f30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1346c8ba4b84f84a1f333c4452157fe', 'request_id': 'req-15094aac-0c9d-4b61-a659-292da6907f30'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:34.389022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-15094aac-0c9d-4b61-a659-292da6907f30 admin admin] The token cache is being invalidate because project d1346c8ba4b84f84a1f333c4452157fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:34.389576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-15094aac-0c9d-4b61-a659-292da6907f30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:34.390538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-15094aac-0c9d-4b61-a659-292da6907f30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:34.392462 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1050/2097] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:34 2022] DELETE /identity/v3/projects/d1346c8ba4b84f84a1f333c4452157fe => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:56:34.397827 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43491b65-a66b-48c8-b4e1-64acfe4faf4d service glance] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:34.424807 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43491b65-a66b-48c8-b4e1-64acfe4faf4d service glance] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:34.500205 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a93af9-19fb-44ac-9d0f-5d0bd0a08cc4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:34.500755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a93af9-19fb-44ac-9d0f-5d0bd0a08cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:34.501046 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a93af9-19fb-44ac-9d0f-5d0bd0a08cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:34.538754 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7809ef-45c7-41c2-875b-343bec4be047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:34.539860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7809ef-45c7-41c2-875b-343bec4be047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:34.540017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7809ef-45c7-41c2-875b-343bec4be047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:34.540353 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7809ef-45c7-41c2-875b-343bec4be047 admin admin] PATH_INFO: `/v3/users/47335afeca6044e298509414f5cc1910` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:34.541823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7809ef-45c7-41c2-875b-343bec4be047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47335afeca6044e298509414f5cc1910)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:34.543899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7809ef-45c7-41c2-875b-343bec4be047 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:34.573290 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:34.574423 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d4a93af9-19fb-44ac-9d0f-5d0bd0a08cc4 None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:34.577410 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d4a93af9-19fb-44ac-9d0f-5d0bd0a08cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:34.671821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9c7809ef-45c7-41c2-875b-343bec4be047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47335afeca6044e298509414f5cc1910'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:34.672563 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9c7809ef-45c7-41c2-875b-343bec4be047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47335afeca6044e298509414f5cc1910'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:34.699903 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9c7809ef-45c7-41c2-875b-343bec4be047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47335afeca6044e298509414f5cc1910'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:34.707352 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9c7809ef-45c7-41c2-875b-343bec4be047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47335afeca6044e298509414f5cc1910'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:34.720722 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9c7809ef-45c7-41c2-875b-343bec4be047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47335afeca6044e298509414f5cc1910'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:34.742826 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1048/2098] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:34 2022] DELETE /identity/v3/users/47335afeca6044e298509414f5cc1910 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:34.743562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7809ef-45c7-41c2-875b-343bec4be047 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:34.752901 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1051/2099] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Sep 7 07:56:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:56:34.753661 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9c7809ef-45c7-41c2-875b-343bec4be047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:34.863450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a93af9-19fb-44ac-9d0f-5d0bd0a08cc4 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:34.865140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d4a93af9-19fb-44ac-9d0f-5d0bd0a08cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:34.919927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7809ef-45c7-41c2-875b-343bec4be047 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:34.930797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27452bc-6280-4077-ab85-36327abf9114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:34.931553 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27452bc-6280-4077-ab85-36327abf9114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:34.931877 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27452bc-6280-4077-ab85-36327abf9114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:34.932631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27452bc-6280-4077-ab85-36327abf9114 admin admin] PATH_INFO: `/v3/projects/9cb2dc5621a645d8853b76441c3e018e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:34.937317 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:34.945184 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27452bc-6280-4077-ab85-36327abf9114 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cb2dc5621a645d8853b76441c3e018e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:34.949184 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27452bc-6280-4077-ab85-36327abf9114 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:34.993008 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a93af9-19fb-44ac-9d0f-5d0bd0a08cc4 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:35.032910 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f27452bc-6280-4077-ab85-36327abf9114 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cb2dc5621a645d8853b76441c3e018e', 'request_id': 'req-f27452bc-6280-4077-ab85-36327abf9114'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:35.048666 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f27452bc-6280-4077-ab85-36327abf9114 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cb2dc5621a645d8853b76441c3e018e', 'request_id': 'req-f27452bc-6280-4077-ab85-36327abf9114'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:35.060645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f27452bc-6280-4077-ab85-36327abf9114 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cb2dc5621a645d8853b76441c3e018e', 'request_id': 'req-f27452bc-6280-4077-ab85-36327abf9114'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:35.082670 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f27452bc-6280-4077-ab85-36327abf9114 admin admin] The token cache is being invalidate because project 9cb2dc5621a645d8853b76441c3e018e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:35.082981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f27452bc-6280-4077-ab85-36327abf9114 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:35.083653 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f27452bc-6280-4077-ab85-36327abf9114 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:35.089280 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1049/2100] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:34 2022] DELETE /identity/v3/projects/9cb2dc5621a645d8853b76441c3e018e => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:35.091969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2418c1-c937-4c08-a0a3-5b2dab2d37bb None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:35.092305 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2418c1-c937-4c08-a0a3-5b2dab2d37bb None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:35.092587 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2418c1-c937-4c08-a0a3-5b2dab2d37bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:35.102874 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc321a-abe5-4116-8deb-467cebf4d235 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:35.103629 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc321a-abe5-4116-8deb-467cebf4d235 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:35.104010 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc321a-abe5-4116-8deb-467cebf4d235 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:35.104454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc321a-abe5-4116-8deb-467cebf4d235 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:35.106410 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c5dc321a-abe5-4116-8deb-467cebf4d235 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:35.129203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-4a2418c1-c937-4c08-a0a3-5b2dab2d37bb None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:35.130823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4a2418c1-c937-4c08-a0a3-5b2dab2d37bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:35.149571 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:35.206737 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc321a-abe5-4116-8deb-467cebf4d235 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:35.208861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc321a-abe5-4116-8deb-467cebf4d235 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:35.229087 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1052/2101] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 07:56:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 07:56:35.230126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc321a-abe5-4116-8deb-467cebf4d235 service swift] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:35.231343 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c5dc321a-abe5-4116-8deb-467cebf4d235 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:35.249606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1050/2102] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:35.249822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc321a-abe5-4116-8deb-467cebf4d235 service swift] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:35.254650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81079129-415d-470a-85f1-5e8b4190d3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:35.255255 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81079129-415d-470a-85f1-5e8b4190d3ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:35.255530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81079129-415d-470a-85f1-5e8b4190d3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:35.255801 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81079129-415d-470a-85f1-5e8b4190d3ca admin admin] PATH_INFO: `/v3/projects/d4f1d90ef7fa4606890bea7b35c94538` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:35.263407 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a4878-ca88-4ca0-8daf-25de11a842a6 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:35.263407 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a4878-ca88-4ca0-8daf-25de11a842a6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:35.263407 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a4878-ca88-4ca0-8daf-25de11a842a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:35.263823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81079129-415d-470a-85f1-5e8b4190d3ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4f1d90ef7fa4606890bea7b35c94538)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:35.263823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81079129-415d-470a-85f1-5e8b4190d3ca admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:35.287203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-765a4878-ca88-4ca0-8daf-25de11a842a6 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:35.289093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-765a4878-ca88-4ca0-8daf-25de11a842a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:35.301255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1051/2103] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:35.302568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a4878-ca88-4ca0-8daf-25de11a842a6 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:35.318037 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a4878-ca88-4ca0-8daf-25de11a842a6 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:35.325972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba14f1-8e4c-498c-975f-4a973bb5c0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:35.326585 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba14f1-8e4c-498c-975f-4a973bb5c0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:35.327048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba14f1-8e4c-498c-975f-4a973bb5c0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:35.330303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba14f1-8e4c-498c-975f-4a973bb5c0c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:35.331184 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba14f1-8e4c-498c-975f-4a973bb5c0c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:56:35.331491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba14f1-8e4c-498c-975f-4a973bb5c0c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:35.333507 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba14f1-8e4c-498c-975f-4a973bb5c0c0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:35.352969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1052/2104] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:56:35.354872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba14f1-8e4c-498c-975f-4a973bb5c0c0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:35.367023 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba14f1-8e4c-498c-975f-4a973bb5c0c0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:35.367633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81079129-415d-470a-85f1-5e8b4190d3ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4f1d90ef7fa4606890bea7b35c94538', 'request_id': 'req-81079129-415d-470a-85f1-5e8b4190d3ca'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:35.387841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81079129-415d-470a-85f1-5e8b4190d3ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4f1d90ef7fa4606890bea7b35c94538', 'request_id': 'req-81079129-415d-470a-85f1-5e8b4190d3ca'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:35.395938 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81079129-415d-470a-85f1-5e8b4190d3ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4f1d90ef7fa4606890bea7b35c94538', 'request_id': 'req-81079129-415d-470a-85f1-5e8b4190d3ca'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:35.412099 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81079129-415d-470a-85f1-5e8b4190d3ca admin admin] The token cache is being invalidate because project d4f1d90ef7fa4606890bea7b35c94538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:35.412712 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81079129-415d-470a-85f1-5e8b4190d3ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:35.418919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81079129-415d-470a-85f1-5e8b4190d3ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:35.421679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1053/2105] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:35 2022] DELETE /identity/v3/projects/d4f1d90ef7fa4606890bea7b35c94538 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:35.429366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaad26a-b3aa-40e8-be62-42e49a984406 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:35.429570 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaad26a-b3aa-40e8-be62-42e49a984406 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:35.429857 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaad26a-b3aa-40e8-be62-42e49a984406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:35.441468 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:35.454670 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c6fdd-447a-47f7-bf4d-005911b6d83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:35.455671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c6fdd-447a-47f7-bf4d-005911b6d83e admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:35.456220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c6fdd-447a-47f7-bf4d-005911b6d83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:35.456745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c6fdd-447a-47f7-bf4d-005911b6d83e admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:35.457585 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c6fdd-447a-47f7-bf4d-005911b6d83e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:35.458702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cbaad26a-b3aa-40e8-be62-42e49a984406 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:35.460220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c6fdd-447a-47f7-bf4d-005911b6d83e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:35.460642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cbaad26a-b3aa-40e8-be62-42e49a984406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:35.476472 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1054/2106] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:35.486902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04046b9c-190e-47ee-ab15-3586a76075b6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:35.487519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04046b9c-190e-47ee-ab15-3586a76075b6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:35.487798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04046b9c-190e-47ee-ab15-3586a76075b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:35.505797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1053/2107] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:35 2022] POST /identity/v3/projects => generated 369 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:35.511481 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c6fdd-447a-47f7-bf4d-005911b6d83e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:35.513494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c48c6fdd-447a-47f7-bf4d-005911b6d83e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:35.514933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-04046b9c-190e-47ee-ab15-3586a76075b6 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:35.516188 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-04046b9c-190e-47ee-ab15-3586a76075b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:35.620096 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1055/2108] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:35.623739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04046b9c-190e-47ee-ab15-3586a76075b6 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:35.635644 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c6fdd-447a-47f7-bf4d-005911b6d83e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:35.637504 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04046b9c-190e-47ee-ab15-3586a76075b6 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:35.642708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b5356-cfb7-4586-8abb-983c8a5305c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:35.643784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b5356-cfb7-4586-8abb-983c8a5305c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:35.644303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b5356-cfb7-4586-8abb-983c8a5305c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:35.644782 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b5356-cfb7-4586-8abb-983c8a5305c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:35.645732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b5356-cfb7-4586-8abb-983c8a5305c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:35.646463 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c419af-4640-4252-a9a1-40ff4c009aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:35.648563 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c419af-4640-4252-a9a1-40ff4c009aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:35.648850 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c419af-4640-4252-a9a1-40ff4c009aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:35.649161 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c419af-4640-4252-a9a1-40ff4c009aa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:35.650165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b5356-cfb7-4586-8abb-983c8a5305c3 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:35.652945 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c419af-4640-4252-a9a1-40ff4c009aa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:56:35.653384 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c419af-4640-4252-a9a1-40ff4c009aa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:35.655130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c419af-4640-4252-a9a1-40ff4c009aa7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:35.685946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1056/2109] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:56:35.692364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c419af-4640-4252-a9a1-40ff4c009aa7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:35.706740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1054/2110] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:35 2022] POST /identity/v3/users => generated 285 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:35.707444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c419af-4640-4252-a9a1-40ff4c009aa7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:35.711476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b5356-cfb7-4586-8abb-983c8a5305c3 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:35.714320 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af8680-401c-4d04-a4fe-8e13301ab485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:35.714603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af8680-401c-4d04-a4fe-8e13301ab485 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:35.715598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af8680-401c-4d04-a4fe-8e13301ab485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:35.715931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af8680-401c-4d04-a4fe-8e13301ab485 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:35.716806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af8680-401c-4d04-a4fe-8e13301ab485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:35.725411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af8680-401c-4d04-a4fe-8e13301ab485 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:35.726145 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b5356-cfb7-4586-8abb-983c8a5305c3 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:35.731788 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58461f64-806e-41b9-a845-99d56f93bf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:35.732442 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58461f64-806e-41b9-a845-99d56f93bf2d admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:35.732725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58461f64-806e-41b9-a845-99d56f93bf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:35.732992 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58461f64-806e-41b9-a845-99d56f93bf2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:35.733635 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58461f64-806e-41b9-a845-99d56f93bf2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:35.735681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58461f64-806e-41b9-a845-99d56f93bf2d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:35.751012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1055/2111] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:35.752056 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58461f64-806e-41b9-a845-99d56f93bf2d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:35.764402 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:35.769887 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58461f64-806e-41b9-a845-99d56f93bf2d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:35.772294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1057/2112] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:35 2022] POST /identity/v3/projects => generated 367 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:56:35.778022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af8680-401c-4d04-a4fe-8e13301ab485 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:35.925849 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48092efa-6968-4f27-9f4a-0277e3b83075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:35.926353 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48092efa-6968-4f27-9f4a-0277e3b83075 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:35.926638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48092efa-6968-4f27-9f4a-0277e3b83075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:35.927457 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48092efa-6968-4f27-9f4a-0277e3b83075 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:35.928741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48092efa-6968-4f27-9f4a-0277e3b83075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:35.930426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48092efa-6968-4f27-9f4a-0277e3b83075 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:35.963811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af8680-401c-4d04-a4fe-8e13301ab485 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:35.972411 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1056/2113] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:35.978635 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48092efa-6968-4f27-9f4a-0277e3b83075 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:35.978924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004d226-8abd-46e6-ad54-0e4dcaf35ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:35.979647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004d226-8abd-46e6-ad54-0e4dcaf35ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:35.979933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004d226-8abd-46e6-ad54-0e4dcaf35ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:35.980302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004d226-8abd-46e6-ad54-0e4dcaf35ed1 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:35.982425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004d226-8abd-46e6-ad54-0e4dcaf35ed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:35.984441 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004d226-8abd-46e6-ad54-0e4dcaf35ed1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:36.000299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48092efa-6968-4f27-9f4a-0277e3b83075 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:36.009959 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113c70f-e43d-47cb-a930-1ddecafc1845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:36.012492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113c70f-e43d-47cb-a930-1ddecafc1845 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:36.013613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113c70f-e43d-47cb-a930-1ddecafc1845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:36.016383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1058/2114] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:35 2022] POST /identity/v3/users => generated 284 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:36.016769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113c70f-e43d-47cb-a930-1ddecafc1845 admin admin] PATH_INFO: `/v3/projects/118f441eca754d858a8820d433d5c916/users/975dceb0432e428aa212effd6cff4132/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:36.021050 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004d226-8abd-46e6-ad54-0e4dcaf35ed1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:36.035357 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004d226-8abd-46e6-ad54-0e4dcaf35ed1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:36.065912 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113c70f-e43d-47cb-a930-1ddecafc1845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=118f441eca754d858a8820d433d5c916, user_id=975dceb0432e428aa212effd6cff4132, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:36.070419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113c70f-e43d-47cb-a930-1ddecafc1845 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:36.097766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2f8d5-1e96-4a4a-bb4b-7d64c36d5033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:36.098312 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2f8d5-1e96-4a4a-bb4b-7d64c36d5033 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:36.098618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2f8d5-1e96-4a4a-bb4b-7d64c36d5033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:36.098876 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2f8d5-1e96-4a4a-bb4b-7d64c36d5033 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:36.099792 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2f8d5-1e96-4a4a-bb4b-7d64c36d5033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:36.103056 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2f8d5-1e96-4a4a-bb4b-7d64c36d5033 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:36.108802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1057/2115] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:35 2022] PUT /identity/v3/projects/118f441eca754d858a8820d433d5c916/users/975dceb0432e428aa212effd6cff4132/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:36.121322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45c59a-08ba-4619-a5ec-756d0d612d9c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:36.121649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45c59a-08ba-4619-a5ec-756d0d612d9c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:36.121873 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45c59a-08ba-4619-a5ec-756d0d612d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:36.123677 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1059/2116] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:36.127568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2f8d5-1e96-4a4a-bb4b-7d64c36d5033 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:36.143952 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2f8d5-1e96-4a4a-bb4b-7d64c36d5033 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:36.160927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7e45c59a-08ba-4619-a5ec-756d0d612d9c None None] MFA Rules not processed for user `975dceb0432e428aa212effd6cff4132`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:36.162238 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7e45c59a-08ba-4619-a5ec-756d0d612d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:36.224228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:36.252922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359fef7-8b74-4e81-8509-e3a6f00399a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:36.257848 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359fef7-8b74-4e81-8509-e3a6f00399a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:36.258053 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359fef7-8b74-4e81-8509-e3a6f00399a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:36.258330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359fef7-8b74-4e81-8509-e3a6f00399a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:36.259001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359fef7-8b74-4e81-8509-e3a6f00399a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:36.260632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359fef7-8b74-4e81-8509-e3a6f00399a9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:36.279931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1060/2117] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:56:36.290233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359fef7-8b74-4e81-8509-e3a6f00399a9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:36.319715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359fef7-8b74-4e81-8509-e3a6f00399a9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:36.325758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1058/2118] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:36.332620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec11977-a837-4018-a409-c3e678e54360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:36.333307 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec11977-a837-4018-a409-c3e678e54360 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:36.333581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec11977-a837-4018-a409-c3e678e54360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:36.333856 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec11977-a837-4018-a409-c3e678e54360 admin admin] PATH_INFO: `/v3/projects/c5aa5def01ac460c96896c429754ba38/users/1bd6ca8815234de8b70d3bb4a581c1a3/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:36.339561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45c59a-08ba-4619-a5ec-756d0d612d9c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:36.342652 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7e45c59a-08ba-4619-a5ec-756d0d612d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:36.376350 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec11977-a837-4018-a409-c3e678e54360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5aa5def01ac460c96896c429754ba38, user_id=1bd6ca8815234de8b70d3bb4a581c1a3, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:36.382277 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec11977-a837-4018-a409-c3e678e54360 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:36.410719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1061/2119] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:36 2022] PUT /identity/v3/projects/c5aa5def01ac460c96896c429754ba38/users/1bd6ca8815234de8b70d3bb4a581c1a3/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:36.429940 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85608b5-b419-4153-9fa1-f0986102b5c1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:36.431618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85608b5-b419-4153-9fa1-f0986102b5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:36.433073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85608b5-b419-4153-9fa1-f0986102b5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:36.468802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45c59a-08ba-4619-a5ec-756d0d612d9c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:36.486756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c85608b5-b419-4153-9fa1-f0986102b5c1 None None] MFA Rules not processed for user `1bd6ca8815234de8b70d3bb4a581c1a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:36.487998 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c85608b5-b419-4153-9fa1-f0986102b5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:36.492998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:36.564541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6558a0-a967-4331-96cf-dadda841e112 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b0652c89c9945e2866512c9da4793ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:36.567875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6558a0-a967-4331-96cf-dadda841e112 service glance] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:36.568972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6558a0-a967-4331-96cf-dadda841e112 service glance] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:36.569348 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6558a0-a967-4331-96cf-dadda841e112 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:36.586528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1062/2120] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:56:36.589207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85608b5-b419-4153-9fa1-f0986102b5c1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:36.703832 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85608b5-b419-4153-9fa1-f0986102b5c1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:36.707212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a0d9a-cfe0-4ec3-bdec-c4c4cdf69736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:36.707859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a0d9a-cfe0-4ec3-bdec-c4c4cdf69736 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:36.708432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a0d9a-cfe0-4ec3-bdec-c4c4cdf69736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:36.708432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a0d9a-cfe0-4ec3-bdec-c4c4cdf69736 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:36.709083 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a0d9a-cfe0-4ec3-bdec-c4c4cdf69736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:36.711532 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a0d9a-cfe0-4ec3-bdec-c4c4cdf69736 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:36.713116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6558a0-a967-4331-96cf-dadda841e112 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:36.715194 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6558a0-a967-4331-96cf-dadda841e112 service glance] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:36.732242 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1059/2121] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 07:56:36.761308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1063/2122] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:36 2022] POST /identity/v3/projects => generated 367 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:36.765844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6558a0-a967-4331-96cf-dadda841e112 service glance] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:36.784942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6558a0-a967-4331-96cf-dadda841e112 service glance] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:36.867978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ce2db-ff42-44ee-9900-b39e71f930eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:36.868611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ce2db-ff42-44ee-9900-b39e71f930eb admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:36.868966 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ce2db-ff42-44ee-9900-b39e71f930eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:36.869468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ce2db-ff42-44ee-9900-b39e71f930eb admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:36.870111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ce2db-ff42-44ee-9900-b39e71f930eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:36.872276 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ce2db-ff42-44ee-9900-b39e71f930eb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:36.935561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a0d9a-cfe0-4ec3-bdec-c4c4cdf69736 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:36.944809 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1060/2123] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:36 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:56:36.950317 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a0d9a-cfe0-4ec3-bdec-c4c4cdf69736 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:36.958639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b4284-1241-4685-8f36-9c7a5c1a3af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:36.960541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b4284-1241-4685-8f36-9c7a5c1a3af1 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:36.960826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b4284-1241-4685-8f36-9c7a5c1a3af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:36.961097 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b4284-1241-4685-8f36-9c7a5c1a3af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:36.961921 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b4284-1241-4685-8f36-9c7a5c1a3af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:36.963530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b4284-1241-4685-8f36-9c7a5c1a3af1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:37.010001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1064/2124] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:56:37.011286 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ce2db-ff42-44ee-9900-b39e71f930eb admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:37.024300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ce2db-ff42-44ee-9900-b39e71f930eb admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:37.040217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb30d7-e492-454d-a80b-b3877fdacd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:37.040892 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb30d7-e492-454d-a80b-b3877fdacd15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:37.041204 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb30d7-e492-454d-a80b-b3877fdacd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:37.041604 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb30d7-e492-454d-a80b-b3877fdacd15 admin admin] PATH_INFO: `/v3/projects/470bc7ff2e494e65bbe57d1f45e2669a/users/8f5054e7f824424099cb66888a6bedc1/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:37.077889 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb30d7-e492-454d-a80b-b3877fdacd15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=470bc7ff2e494e65bbe57d1f45e2669a, user_id=8f5054e7f824424099cb66888a6bedc1, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:37.080880 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb30d7-e492-454d-a80b-b3877fdacd15 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:37.109682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1061/2125] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:37 2022] PUT /identity/v3/projects/470bc7ff2e494e65bbe57d1f45e2669a/users/8f5054e7f824424099cb66888a6bedc1/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:37.111401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b4284-1241-4685-8f36-9c7a5c1a3af1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:37.125126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b4284-1241-4685-8f36-9c7a5c1a3af1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:37.143100 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:37.246613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56691ac6-373a-4b29-a5a3-945273816237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:37.247310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56691ac6-373a-4b29-a5a3-945273816237 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:37.247597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56691ac6-373a-4b29-a5a3-945273816237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:37.247876 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56691ac6-373a-4b29-a5a3-945273816237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:37.248888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56691ac6-373a-4b29-a5a3-945273816237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:37.250509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56691ac6-373a-4b29-a5a3-945273816237 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:37.279828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb30d7-e492-454d-a80b-b3877fdacd15 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:37.284392 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1065/2126] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:56:37.300324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb30d7-e492-454d-a80b-b3877fdacd15 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:37.342099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ddeaee-ef11-4a47-826d-6cfd5c445027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:37.342900 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ddeaee-ef11-4a47-826d-6cfd5c445027 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:37.350815 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ddeaee-ef11-4a47-826d-6cfd5c445027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:37.350815 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ddeaee-ef11-4a47-826d-6cfd5c445027 admin admin] PATH_INFO: `/v3/projects/470bc7ff2e494e65bbe57d1f45e2669a/users/8f5054e7f824424099cb66888a6bedc1/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:37.355938 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ddeaee-ef11-4a47-826d-6cfd5c445027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=470bc7ff2e494e65bbe57d1f45e2669a, user_id=8f5054e7f824424099cb66888a6bedc1, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:37.358924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ddeaee-ef11-4a47-826d-6cfd5c445027 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:37.390227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1062/2127] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:37 2022] PUT /identity/v3/projects/470bc7ff2e494e65bbe57d1f45e2669a/users/8f5054e7f824424099cb66888a6bedc1/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:37.402659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f3e50-e99a-43b7-8993-325b8b25ca7c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:37.402979 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f3e50-e99a-43b7-8993-325b8b25ca7c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:37.405682 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f3e50-e99a-43b7-8993-325b8b25ca7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:37.547574 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ce4f3e50-e99a-43b7-8993-325b8b25ca7c None None] MFA Rules not processed for user `8f5054e7f824424099cb66888a6bedc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:37.549151 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ce4f3e50-e99a-43b7-8993-325b8b25ca7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:37.663380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1066/2128] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:37.675256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ddeaee-ef11-4a47-826d-6cfd5c445027 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:37.677045 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-22ddeaee-ef11-4a47-826d-6cfd5c445027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:37.720603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:37.776283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ddeaee-ef11-4a47-826d-6cfd5c445027 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:37.781057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98faf9f-63f5-4f61-ad56-c838f777cdac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:37.781666 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98faf9f-63f5-4f61-ad56-c838f777cdac service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:37.781940 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98faf9f-63f5-4f61-ad56-c838f777cdac service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:37.782207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98faf9f-63f5-4f61-ad56-c838f777cdac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:37.801247 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98faf9f-63f5-4f61-ad56-c838f777cdac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:37.803275 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98faf9f-63f5-4f61-ad56-c838f777cdac service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:37.822057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1063/2129] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:37 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:37.944948 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f3e50-e99a-43b7-8993-325b8b25ca7c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:37.956030 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f3e50-e99a-43b7-8993-325b8b25ca7c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:37.963373 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba15a4f-f799-4fd3-9249-77096ac871f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:37.964006 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba15a4f-f799-4fd3-9249-77096ac871f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:37.964301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba15a4f-f799-4fd3-9249-77096ac871f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:37.964576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba15a4f-f799-4fd3-9249-77096ac871f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:38.053736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba15a4f-f799-4fd3-9249-77096ac871f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:38.058928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba15a4f-f799-4fd3-9249-77096ac871f7 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:38.082489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1067/2130] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:43.853208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98faf9f-63f5-4f61-ad56-c838f777cdac service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:44.067409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98faf9f-63f5-4f61-ad56-c838f777cdac service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:44.074230 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa7df8-b48e-4f34-910e-da0f9a7f6515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:44.075232 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa7df8-b48e-4f34-910e-da0f9a7f6515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:44.075860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa7df8-b48e-4f34-910e-da0f9a7f6515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:44.076377 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa7df8-b48e-4f34-910e-da0f9a7f6515 admin admin] PATH_INFO: `/v3/users/1bd6ca8815234de8b70d3bb4a581c1a3` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:44.077898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa7df8-b48e-4f34-910e-da0f9a7f6515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bd6ca8815234de8b70d3bb4a581c1a3)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:44.080112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa7df8-b48e-4f34-910e-da0f9a7f6515 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:44.217305 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-66aa7df8-b48e-4f34-910e-da0f9a7f6515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bd6ca8815234de8b70d3bb4a581c1a3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:44.218077 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-66aa7df8-b48e-4f34-910e-da0f9a7f6515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bd6ca8815234de8b70d3bb4a581c1a3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:44.233541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-66aa7df8-b48e-4f34-910e-da0f9a7f6515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bd6ca8815234de8b70d3bb4a581c1a3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:44.234270 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-66aa7df8-b48e-4f34-910e-da0f9a7f6515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bd6ca8815234de8b70d3bb4a581c1a3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:44.250017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-66aa7df8-b48e-4f34-910e-da0f9a7f6515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bd6ca8815234de8b70d3bb4a581c1a3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:44.282240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1064/2131] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:43 2022] DELETE /identity/v3/users/1bd6ca8815234de8b70d3bb4a581c1a3 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:44.286550 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe88306-6ed8-4d27-98a3-08052b8eb14e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:44.287119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe88306-6ed8-4d27-98a3-08052b8eb14e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:44.287534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe88306-6ed8-4d27-98a3-08052b8eb14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:44.340022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-fbe88306-6ed8-4d27-98a3-08052b8eb14e None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:44.341722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fbe88306-6ed8-4d27-98a3-08052b8eb14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:44.407964 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:44.456432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1068/2132] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:56:44.467307 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa7df8-b48e-4f34-910e-da0f9a7f6515 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:44.469061 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-66aa7df8-b48e-4f34-910e-da0f9a7f6515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:44.549558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa7df8-b48e-4f34-910e-da0f9a7f6515 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:44.553709 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbe947-2eeb-418f-9b3f-9455880cde6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:44.554308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbe947-2eeb-418f-9b3f-9455880cde6e service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:44.554587 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbe947-2eeb-418f-9b3f-9455880cde6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:44.554855 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbe947-2eeb-418f-9b3f-9455880cde6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:44.561053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:44.565111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbe947-2eeb-418f-9b3f-9455880cde6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:44.570847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbe947-2eeb-418f-9b3f-9455880cde6e service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:44.582790 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1065/2133] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 07:56:44.985487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe88306-6ed8-4d27-98a3-08052b8eb14e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:44.987429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fbe88306-6ed8-4d27-98a3-08052b8eb14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:45.000422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe88306-6ed8-4d27-98a3-08052b8eb14e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:45.029950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f593f-ce06-4410-a921-1658f0678f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:45.030588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f593f-ce06-4410-a921-1658f0678f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:45.030946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f593f-ce06-4410-a921-1658f0678f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:45.031375 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f593f-ce06-4410-a921-1658f0678f36 admin admin] PATH_INFO: `/v3/projects/c5aa5def01ac460c96896c429754ba38` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:45.040838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f593f-ce06-4410-a921-1658f0678f36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5aa5def01ac460c96896c429754ba38)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:45.062924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f593f-ce06-4410-a921-1658f0678f36 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:45.135517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c66f593f-ce06-4410-a921-1658f0678f36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5aa5def01ac460c96896c429754ba38', 'request_id': 'req-c66f593f-ce06-4410-a921-1658f0678f36'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:45.142233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c66f593f-ce06-4410-a921-1658f0678f36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5aa5def01ac460c96896c429754ba38', 'request_id': 'req-c66f593f-ce06-4410-a921-1658f0678f36'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:45.150942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c66f593f-ce06-4410-a921-1658f0678f36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5aa5def01ac460c96896c429754ba38', 'request_id': 'req-c66f593f-ce06-4410-a921-1658f0678f36'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:45.166353 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c66f593f-ce06-4410-a921-1658f0678f36 admin admin] The token cache is being invalidate because project c5aa5def01ac460c96896c429754ba38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:45.166687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c66f593f-ce06-4410-a921-1658f0678f36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:45.167289 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c66f593f-ce06-4410-a921-1658f0678f36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:45.169060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1069/2134] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:44 2022] DELETE /identity/v3/projects/c5aa5def01ac460c96896c429754ba38 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:45.173466 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbe947-2eeb-418f-9b3f-9455880cde6e service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:45.186162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbe947-2eeb-418f-9b3f-9455880cde6e service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:45.266033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09e711-affc-4b31-938f-11e0999ff71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:45.267176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09e711-affc-4b31-938f-11e0999ff71a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:45.267545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09e711-affc-4b31-938f-11e0999ff71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:45.268055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09e711-affc-4b31-938f-11e0999ff71a admin admin] PATH_INFO: `/v3/users/8f5054e7f824424099cb66888a6bedc1` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:45.269461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09e711-affc-4b31-938f-11e0999ff71a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f5054e7f824424099cb66888a6bedc1)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:45.284400 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09e711-affc-4b31-938f-11e0999ff71a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:45.387868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bd09e711-affc-4b31-938f-11e0999ff71a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f5054e7f824424099cb66888a6bedc1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:45.398411 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bd09e711-affc-4b31-938f-11e0999ff71a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f5054e7f824424099cb66888a6bedc1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:45.422724 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bd09e711-affc-4b31-938f-11e0999ff71a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f5054e7f824424099cb66888a6bedc1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:45.423764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bd09e711-affc-4b31-938f-11e0999ff71a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f5054e7f824424099cb66888a6bedc1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:45.469552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bd09e711-affc-4b31-938f-11e0999ff71a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f5054e7f824424099cb66888a6bedc1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:45.492071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1066/2135] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:45 2022] DELETE /identity/v3/users/8f5054e7f824424099cb66888a6bedc1 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:45.867437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f593f-ce06-4410-a921-1658f0678f36 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:45.869885 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c66f593f-ce06-4410-a921-1658f0678f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:45.904226 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f593f-ce06-4410-a921-1658f0678f36 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:45.988935 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8690879-02b1-4423-94af-6e455b58ef09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:45.989695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8690879-02b1-4423-94af-6e455b58ef09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:45.990814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8690879-02b1-4423-94af-6e455b58ef09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:45.991247 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8690879-02b1-4423-94af-6e455b58ef09 admin admin] PATH_INFO: `/v3/projects/470bc7ff2e494e65bbe57d1f45e2669a` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:45.992500 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8690879-02b1-4423-94af-6e455b58ef09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=470bc7ff2e494e65bbe57d1f45e2669a)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:45.994491 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8690879-02b1-4423-94af-6e455b58ef09 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:46.057374 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f8690879-02b1-4423-94af-6e455b58ef09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '470bc7ff2e494e65bbe57d1f45e2669a', 'request_id': 'req-f8690879-02b1-4423-94af-6e455b58ef09'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:46.064992 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f8690879-02b1-4423-94af-6e455b58ef09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '470bc7ff2e494e65bbe57d1f45e2669a', 'request_id': 'req-f8690879-02b1-4423-94af-6e455b58ef09'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:46.072701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f8690879-02b1-4423-94af-6e455b58ef09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '470bc7ff2e494e65bbe57d1f45e2669a', 'request_id': 'req-f8690879-02b1-4423-94af-6e455b58ef09'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:46.074409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:46.090094 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f8690879-02b1-4423-94af-6e455b58ef09 admin admin] The token cache is being invalidate because project 470bc7ff2e494e65bbe57d1f45e2669a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:46.090480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f8690879-02b1-4423-94af-6e455b58ef09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:46.091361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f8690879-02b1-4423-94af-6e455b58ef09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:46.093877 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1070/2136] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:45 2022] DELETE /identity/v3/projects/470bc7ff2e494e65bbe57d1f45e2669a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:46.098748 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5cd34-a773-430f-b1a9-7953287712f6 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:46.099031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5cd34-a773-430f-b1a9-7953287712f6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:46.099379 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5cd34-a773-430f-b1a9-7953287712f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:46.126388 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a7d5cd34-a773-430f-b1a9-7953287712f6 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:46.128035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a7d5cd34-a773-430f-b1a9-7953287712f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:46.232915 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1067/2137] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:46.234586 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4fda6e-b195-4504-8ebc-5b7bee762c0a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:46.234913 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4fda6e-b195-4504-8ebc-5b7bee762c0a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:46.235195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4fda6e-b195-4504-8ebc-5b7bee762c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:46.261334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4c4fda6e-b195-4504-8ebc-5b7bee762c0a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:46.264131 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4c4fda6e-b195-4504-8ebc-5b7bee762c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:46.272840 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1071/2138] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:46.277198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5cd34-a773-430f-b1a9-7953287712f6 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:46.291498 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5cd34-a773-430f-b1a9-7953287712f6 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:46.296162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b2813-6158-43f9-9ee9-5c2253c5b1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:46.296811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b2813-6158-43f9-9ee9-5c2253c5b1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:46.297095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b2813-6158-43f9-9ee9-5c2253c5b1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:46.297371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b2813-6158-43f9-9ee9-5c2253c5b1e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:46.298014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b2813-6158-43f9-9ee9-5c2253c5b1e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:56:46.298335 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b2813-6158-43f9-9ee9-5c2253c5b1e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:46.299965 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b2813-6158-43f9-9ee9-5c2253c5b1e9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:46.314572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:46.324797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1068/2139] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:56:46.325903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4fda6e-b195-4504-8ebc-5b7bee762c0a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:46.335954 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4fda6e-b195-4504-8ebc-5b7bee762c0a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:46.341294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e37c59-aa0a-48d0-8577-e33281299284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:46.341932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e37c59-aa0a-48d0-8577-e33281299284 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:46.342211 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e37c59-aa0a-48d0-8577-e33281299284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:46.342508 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e37c59-aa0a-48d0-8577-e33281299284 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:46.343140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e37c59-aa0a-48d0-8577-e33281299284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:46.345078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e37c59-aa0a-48d0-8577-e33281299284 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:46.384164 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1072/2140] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:46 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:56:46.386355 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b2813-6158-43f9-9ee9-5c2253c5b1e9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:46.397751 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b2813-6158-43f9-9ee9-5c2253c5b1e9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:46.504699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42fb50-df3b-4297-bfae-eb43517d45bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:46.506226 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42fb50-df3b-4297-bfae-eb43517d45bb admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:46.507033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42fb50-df3b-4297-bfae-eb43517d45bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:46.508026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42fb50-df3b-4297-bfae-eb43517d45bb admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:46.509090 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42fb50-df3b-4297-bfae-eb43517d45bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:46.511272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42fb50-df3b-4297-bfae-eb43517d45bb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:46.552464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1069/2141] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:46 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:46.553864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e37c59-aa0a-48d0-8577-e33281299284 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:46.565546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e37c59-aa0a-48d0-8577-e33281299284 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:46.574024 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708038ee-66b1-4afb-bee4-f3de8c50774e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:46.574918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708038ee-66b1-4afb-bee4-f3de8c50774e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:46.575364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708038ee-66b1-4afb-bee4-f3de8c50774e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:46.575957 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708038ee-66b1-4afb-bee4-f3de8c50774e admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:46.576829 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708038ee-66b1-4afb-bee4-f3de8c50774e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:46.579156 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708038ee-66b1-4afb-bee4-f3de8c50774e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:46.607849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1073/2142] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:46.612033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42fb50-df3b-4297-bfae-eb43517d45bb admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:46.627066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42fb50-df3b-4297-bfae-eb43517d45bb admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:46.643265 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6f2c5-7100-4614-85b2-15f1596529c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:46.644782 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6f2c5-7100-4614-85b2-15f1596529c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:46.645083 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6f2c5-7100-4614-85b2-15f1596529c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:46.645350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6f2c5-7100-4614-85b2-15f1596529c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:46.646007 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6f2c5-7100-4614-85b2-15f1596529c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:46.653234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6f2c5-7100-4614-85b2-15f1596529c0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:46.680273 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1070/2143] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:46.685519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708038ee-66b1-4afb-bee4-f3de8c50774e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:46.706489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708038ee-66b1-4afb-bee4-f3de8c50774e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:46.713877 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266397f1-47be-449d-910a-34c8d0b5ff4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:46.714754 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266397f1-47be-449d-910a-34c8d0b5ff4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:46.715051 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266397f1-47be-449d-910a-34c8d0b5ff4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:46.715323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266397f1-47be-449d-910a-34c8d0b5ff4c admin admin] PATH_INFO: `/v3/projects/01fcae52127e493486a48a97019096c6/users/5cfa5701ea224e32802824f14c126421/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:46.735061 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266397f1-47be-449d-910a-34c8d0b5ff4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=01fcae52127e493486a48a97019096c6, user_id=5cfa5701ea224e32802824f14c126421, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:46.737772 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266397f1-47be-449d-910a-34c8d0b5ff4c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:46.760415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1074/2144] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:46 2022] PUT /identity/v3/projects/01fcae52127e493486a48a97019096c6/users/5cfa5701ea224e32802824f14c126421/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:46.772583 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78972bff-e972-4b0e-b188-3b26d31b71bb None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:46.772867 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78972bff-e972-4b0e-b188-3b26d31b71bb None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:46.773142 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78972bff-e972-4b0e-b188-3b26d31b71bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:46.806464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-78972bff-e972-4b0e-b188-3b26d31b71bb None None] MFA Rules not processed for user `5cfa5701ea224e32802824f14c126421`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:46.808495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-78972bff-e972-4b0e-b188-3b26d31b71bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:46.920721 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1071/2145] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:46.925545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266397f1-47be-449d-910a-34c8d0b5ff4c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:47.031703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266397f1-47be-449d-910a-34c8d0b5ff4c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:47.041314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c22b8-656e-4589-9dbb-2f8339a24433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:47.041983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c22b8-656e-4589-9dbb-2f8339a24433 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:47.042267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c22b8-656e-4589-9dbb-2f8339a24433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:47.042542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c22b8-656e-4589-9dbb-2f8339a24433 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:47.048137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c22b8-656e-4589-9dbb-2f8339a24433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:47.048137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c22b8-656e-4589-9dbb-2f8339a24433 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:47.122753 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1075/2146] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:46 2022] POST /identity/v3/projects => generated 387 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:47.127413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78972bff-e972-4b0e-b188-3b26d31b71bb None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:47.139079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78972bff-e972-4b0e-b188-3b26d31b71bb None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:47.223586 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890d4b5-9de8-401d-b32b-89db403ededf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:47.224372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890d4b5-9de8-401d-b32b-89db403ededf admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:47.224834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890d4b5-9de8-401d-b32b-89db403ededf admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:47.225341 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890d4b5-9de8-401d-b32b-89db403ededf admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:47.226068 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890d4b5-9de8-401d-b32b-89db403ededf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:47.230294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890d4b5-9de8-401d-b32b-89db403ededf admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:47.262520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:47.269872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1072/2147] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:47 2022] POST /identity/v3/users => generated 300 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:47.274862 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c22b8-656e-4589-9dbb-2f8339a24433 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:47.286764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c22b8-656e-4589-9dbb-2f8339a24433 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:47.294459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999f01d-06a4-45ad-8225-6335698f469e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:47.295074 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999f01d-06a4-45ad-8225-6335698f469e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:47.295471 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999f01d-06a4-45ad-8225-6335698f469e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:47.295750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999f01d-06a4-45ad-8225-6335698f469e admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:47.296421 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999f01d-06a4-45ad-8225-6335698f469e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:47.298061 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999f01d-06a4-45ad-8225-6335698f469e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:47.312138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1076/2148] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:47.315922 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890d4b5-9de8-401d-b32b-89db403ededf admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:47.326976 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890d4b5-9de8-401d-b32b-89db403ededf admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:47.335689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d75c2-4bc2-49d9-bad5-90497f4cc7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:47.336395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d75c2-4bc2-49d9-bad5-90497f4cc7fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:47.336678 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d75c2-4bc2-49d9-bad5-90497f4cc7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:47.336952 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d75c2-4bc2-49d9-bad5-90497f4cc7fb admin admin] PATH_INFO: `/v3/projects/e318bec2f94a4568bdecf05ecf656b72/users/363b479b4a454a14a4d29deea6ad491e/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:47.359540 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d75c2-4bc2-49d9-bad5-90497f4cc7fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e318bec2f94a4568bdecf05ecf656b72, user_id=363b479b4a454a14a4d29deea6ad491e, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:47.362475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d75c2-4bc2-49d9-bad5-90497f4cc7fb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:47.379015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1073/2149] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:47 2022] PUT /identity/v3/projects/e318bec2f94a4568bdecf05ecf656b72/users/363b479b4a454a14a4d29deea6ad491e/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:47.385183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999f01d-06a4-45ad-8225-6335698f469e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:47.399519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999f01d-06a4-45ad-8225-6335698f469e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:47.472220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48938c3d-a71c-41eb-90ef-d9300c28ec2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:47.473136 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48938c3d-a71c-41eb-90ef-d9300c28ec2a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:47.473593 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48938c3d-a71c-41eb-90ef-d9300c28ec2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:47.474029 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48938c3d-a71c-41eb-90ef-d9300c28ec2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:47.474714 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48938c3d-a71c-41eb-90ef-d9300c28ec2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:47.476510 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48938c3d-a71c-41eb-90ef-d9300c28ec2a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:47.479929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:47.494133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1077/2150] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:47.495571 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d75c2-4bc2-49d9-bad5-90497f4cc7fb admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:47.509358 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d75c2-4bc2-49d9-bad5-90497f4cc7fb admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:47.516702 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1d64e-c98f-4e67-837a-d95e938a49d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:47.517577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1d64e-c98f-4e67-837a-d95e938a49d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:47.517858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1d64e-c98f-4e67-837a-d95e938a49d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:47.518129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1d64e-c98f-4e67-837a-d95e938a49d0 admin admin] PATH_INFO: `/v3/projects/e318bec2f94a4568bdecf05ecf656b72/users/363b479b4a454a14a4d29deea6ad491e/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:47.520043 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d1d64e-c98f-4e67-837a-d95e938a49d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e318bec2f94a4568bdecf05ecf656b72, user_id=363b479b4a454a14a4d29deea6ad491e, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:47.523481 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d1d64e-c98f-4e67-837a-d95e938a49d0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:47.536174 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1074/2151] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:47 2022] PUT /identity/v3/projects/e318bec2f94a4568bdecf05ecf656b72/users/363b479b4a454a14a4d29deea6ad491e/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:56:47.547106 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2aea6-2880-44f6-9f74-cee3d5e173f4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:47.547394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2aea6-2880-44f6-9f74-cee3d5e173f4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:47.547668 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2aea6-2880-44f6-9f74-cee3d5e173f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:47.586302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-72e2aea6-2880-44f6-9f74-cee3d5e173f4 None None] MFA Rules not processed for user `363b479b4a454a14a4d29deea6ad491e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:47.587660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-72e2aea6-2880-44f6-9f74-cee3d5e173f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:47.703075 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1078/2152] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:47 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:47.714289 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1d64e-c98f-4e67-837a-d95e938a49d0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:47.716139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-12d1d64e-c98f-4e67-837a-d95e938a49d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:47.780476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1d64e-c98f-4e67-837a-d95e938a49d0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:47.784604 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d84f1-b080-41dc-bc71-c78de7dbb388 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:47.785215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d84f1-b080-41dc-bc71-c78de7dbb388 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:47.785489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d84f1-b080-41dc-bc71-c78de7dbb388 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:47.785753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d84f1-b080-41dc-bc71-c78de7dbb388 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:47.799276 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d84f1-b080-41dc-bc71-c78de7dbb388 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:47.800757 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d84f1-b080-41dc-bc71-c78de7dbb388 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:47.820681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1075/2153] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:47.933475 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708593d-e945-49a2-931f-36fe07df2d06 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:47.933944 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708593d-e945-49a2-931f-36fe07df2d06 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:47.934159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708593d-e945-49a2-931f-36fe07df2d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:47.954662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3708593d-e945-49a2-931f-36fe07df2d06 None None] MFA Rules not processed for user `f018f290f36f4121becdfcc2bc10827c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:47.955866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3708593d-e945-49a2-931f-36fe07df2d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:48.014455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1079/2154] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:56:48.497819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d84f1-b080-41dc-bc71-c78de7dbb388 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:48.567050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d84f1-b080-41dc-bc71-c78de7dbb388 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:48.572582 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89dc81-c0ee-4ffa-8174-332f62cf35ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:48.573217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89dc81-c0ee-4ffa-8174-332f62cf35ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:48.573491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89dc81-c0ee-4ffa-8174-332f62cf35ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:48.573760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89dc81-c0ee-4ffa-8174-332f62cf35ae admin admin] PATH_INFO: `/v3/users/5cfa5701ea224e32802824f14c126421` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:48.576033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89dc81-c0ee-4ffa-8174-332f62cf35ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cfa5701ea224e32802824f14c126421)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:48.576845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89dc81-c0ee-4ffa-8174-332f62cf35ae admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:48.626501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ed89dc81-c0ee-4ffa-8174-332f62cf35ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cfa5701ea224e32802824f14c126421'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:48.627024 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ed89dc81-c0ee-4ffa-8174-332f62cf35ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cfa5701ea224e32802824f14c126421'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:48.640320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ed89dc81-c0ee-4ffa-8174-332f62cf35ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cfa5701ea224e32802824f14c126421'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:48.641059 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ed89dc81-c0ee-4ffa-8174-332f62cf35ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cfa5701ea224e32802824f14c126421'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:48.642471 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:48.649719 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ed89dc81-c0ee-4ffa-8174-332f62cf35ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cfa5701ea224e32802824f14c126421'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:48.663315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1076/2155] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:48 2022] DELETE /identity/v3/users/5cfa5701ea224e32802824f14c126421 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:48.667826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b090e0-ca27-4cdd-beb7-fad0b8063896 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:48.668171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b090e0-ca27-4cdd-beb7-fad0b8063896 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:48.668482 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b090e0-ca27-4cdd-beb7-fad0b8063896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:48.692222 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c5b090e0-ca27-4cdd-beb7-fad0b8063896 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:48.694183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c5b090e0-ca27-4cdd-beb7-fad0b8063896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:48.765218 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1080/2156] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:48.775874 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89dc81-c0ee-4ffa-8174-332f62cf35ae admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:48.777492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ed89dc81-c0ee-4ffa-8174-332f62cf35ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:48.846331 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89dc81-c0ee-4ffa-8174-332f62cf35ae admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:48.855781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943e916-f1d3-436e-9bd6-b189e84dd144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:48.856452 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943e916-f1d3-436e-9bd6-b189e84dd144 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:48.856753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943e916-f1d3-436e-9bd6-b189e84dd144 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:48.857072 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943e916-f1d3-436e-9bd6-b189e84dd144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:48.867228 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943e916-f1d3-436e-9bd6-b189e84dd144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:48.868845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943e916-f1d3-436e-9bd6-b189e84dd144 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:48.879876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1077/2157] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:49.287643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b090e0-ca27-4cdd-beb7-fad0b8063896 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:49.289445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c5b090e0-ca27-4cdd-beb7-fad0b8063896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:49.303436 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b090e0-ca27-4cdd-beb7-fad0b8063896 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:49.314859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04376586-48de-4f70-b989-1c0d8ef1ede4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:49.315524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04376586-48de-4f70-b989-1c0d8ef1ede4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:49.315802 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04376586-48de-4f70-b989-1c0d8ef1ede4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:49.316373 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04376586-48de-4f70-b989-1c0d8ef1ede4 admin admin] PATH_INFO: `/v3/projects/01fcae52127e493486a48a97019096c6` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:49.317597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04376586-48de-4f70-b989-1c0d8ef1ede4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01fcae52127e493486a48a97019096c6)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:49.321193 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04376586-48de-4f70-b989-1c0d8ef1ede4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:49.364311 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:49.399961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-04376586-48de-4f70-b989-1c0d8ef1ede4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01fcae52127e493486a48a97019096c6', 'request_id': 'req-04376586-48de-4f70-b989-1c0d8ef1ede4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:49.406215 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-04376586-48de-4f70-b989-1c0d8ef1ede4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01fcae52127e493486a48a97019096c6', 'request_id': 'req-04376586-48de-4f70-b989-1c0d8ef1ede4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:49.413175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-04376586-48de-4f70-b989-1c0d8ef1ede4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01fcae52127e493486a48a97019096c6', 'request_id': 'req-04376586-48de-4f70-b989-1c0d8ef1ede4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:49.435055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-04376586-48de-4f70-b989-1c0d8ef1ede4 admin admin] The token cache is being invalidate because project 01fcae52127e493486a48a97019096c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:49.435361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-04376586-48de-4f70-b989-1c0d8ef1ede4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:49.436210 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-04376586-48de-4f70-b989-1c0d8ef1ede4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:49.437766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1081/2158] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:49 2022] DELETE /identity/v3/projects/01fcae52127e493486a48a97019096c6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:49.442014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943e916-f1d3-436e-9bd6-b189e84dd144 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:49.451849 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943e916-f1d3-436e-9bd6-b189e84dd144 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:49.519921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea54af3-baf1-4a00-8f39-edd09255bb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:49.521396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea54af3-baf1-4a00-8f39-edd09255bb3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:49.521396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea54af3-baf1-4a00-8f39-edd09255bb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:49.521585 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea54af3-baf1-4a00-8f39-edd09255bb3d admin admin] PATH_INFO: `/v3/users/363b479b4a454a14a4d29deea6ad491e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:49.522712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea54af3-baf1-4a00-8f39-edd09255bb3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=363b479b4a454a14a4d29deea6ad491e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:49.524736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea54af3-baf1-4a00-8f39-edd09255bb3d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:49.592701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7ea54af3-baf1-4a00-8f39-edd09255bb3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '363b479b4a454a14a4d29deea6ad491e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:49.593764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7ea54af3-baf1-4a00-8f39-edd09255bb3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '363b479b4a454a14a4d29deea6ad491e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:49.609022 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7ea54af3-baf1-4a00-8f39-edd09255bb3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '363b479b4a454a14a4d29deea6ad491e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:49.610042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7ea54af3-baf1-4a00-8f39-edd09255bb3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '363b479b4a454a14a4d29deea6ad491e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:49.618647 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7ea54af3-baf1-4a00-8f39-edd09255bb3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '363b479b4a454a14a4d29deea6ad491e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:49.635047 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1078/2159] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:49 2022] DELETE /identity/v3/users/363b479b4a454a14a4d29deea6ad491e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:49.941777 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04376586-48de-4f70-b989-1c0d8ef1ede4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:49.944223 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-04376586-48de-4f70-b989-1c0d8ef1ede4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:49.957830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04376586-48de-4f70-b989-1c0d8ef1ede4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:50.072946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5e02e-7763-4c55-b6d4-1ad459fcea7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:50.075077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5e02e-7763-4c55-b6d4-1ad459fcea7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:50.075077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5e02e-7763-4c55-b6d4-1ad459fcea7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:50.075077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5e02e-7763-4c55-b6d4-1ad459fcea7f admin admin] PATH_INFO: `/v3/projects/e318bec2f94a4568bdecf05ecf656b72` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:50.078347 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5e02e-7763-4c55-b6d4-1ad459fcea7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e318bec2f94a4568bdecf05ecf656b72)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:50.078347 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5e02e-7763-4c55-b6d4-1ad459fcea7f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:50.140963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e2a5e02e-7763-4c55-b6d4-1ad459fcea7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e318bec2f94a4568bdecf05ecf656b72', 'request_id': 'req-e2a5e02e-7763-4c55-b6d4-1ad459fcea7f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:50.148396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e2a5e02e-7763-4c55-b6d4-1ad459fcea7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e318bec2f94a4568bdecf05ecf656b72', 'request_id': 'req-e2a5e02e-7763-4c55-b6d4-1ad459fcea7f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:50.155910 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e2a5e02e-7763-4c55-b6d4-1ad459fcea7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e318bec2f94a4568bdecf05ecf656b72', 'request_id': 'req-e2a5e02e-7763-4c55-b6d4-1ad459fcea7f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:50.171563 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e2a5e02e-7763-4c55-b6d4-1ad459fcea7f admin admin] The token cache is being invalidate because project e318bec2f94a4568bdecf05ecf656b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:50.172036 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e2a5e02e-7763-4c55-b6d4-1ad459fcea7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:50.172761 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e2a5e02e-7763-4c55-b6d4-1ad459fcea7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:50.174571 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1082/2160] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:49 2022] DELETE /identity/v3/projects/e318bec2f94a4568bdecf05ecf656b72 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:50.179810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3f6c5-fe58-429f-9be7-27f9f5f9f8ad None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:50.180156 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3f6c5-fe58-429f-9be7-27f9f5f9f8ad None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:50.180441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3f6c5-fe58-429f-9be7-27f9f5f9f8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:50.190844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:50.214511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b0b3f6c5-fe58-429f-9be7-27f9f5f9f8ad None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:50.215764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b0b3f6c5-fe58-429f-9be7-27f9f5f9f8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:50.308850 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1079/2161] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:56:50.317183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f2426-4590-4a72-9a32-f055d3518a94 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:50.317662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f2426-4590-4a72-9a32-f055d3518a94 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:50.318147 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f2426-4590-4a72-9a32-f055d3518a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:50.343840 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cf7f2426-4590-4a72-9a32-f055d3518a94 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:50.345604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cf7f2426-4590-4a72-9a32-f055d3518a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:50.359794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1083/2162] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:50.364014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3f6c5-fe58-429f-9be7-27f9f5f9f8ad None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:50.375568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3f6c5-fe58-429f-9be7-27f9f5f9f8ad None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:50.385060 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf973d28-88e0-4a1f-84a0-24287577d2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:50.385765 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf973d28-88e0-4a1f-84a0-24287577d2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:50.386113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf973d28-88e0-4a1f-84a0-24287577d2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:50.386454 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf973d28-88e0-4a1f-84a0-24287577d2f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:50.387244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf973d28-88e0-4a1f-84a0-24287577d2f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:56:50.387695 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf973d28-88e0-4a1f-84a0-24287577d2f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:50.389705 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf973d28-88e0-4a1f-84a0-24287577d2f9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:50.405020 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1080/2163] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 07:56:50.411572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f2426-4590-4a72-9a32-f055d3518a94 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:50.426162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f2426-4590-4a72-9a32-f055d3518a94 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:50.439482 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610f8c3-4e13-4bcb-812e-2f2224962726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:50.440145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610f8c3-4e13-4bcb-812e-2f2224962726 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:50.440434 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610f8c3-4e13-4bcb-812e-2f2224962726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:50.440910 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610f8c3-4e13-4bcb-812e-2f2224962726 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:50.441964 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610f8c3-4e13-4bcb-812e-2f2224962726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:50.445208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610f8c3-4e13-4bcb-812e-2f2224962726 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:50.486340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1084/2164] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:50 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:50.490827 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf973d28-88e0-4a1f-84a0-24287577d2f9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:50.503432 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf973d28-88e0-4a1f-84a0-24287577d2f9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:50.585931 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea41e66-bcda-4b2b-89d6-32fe4cc0fa64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:50.586706 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea41e66-bcda-4b2b-89d6-32fe4cc0fa64 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:50.587017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea41e66-bcda-4b2b-89d6-32fe4cc0fa64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:50.591896 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea41e66-bcda-4b2b-89d6-32fe4cc0fa64 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:50.591896 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea41e66-bcda-4b2b-89d6-32fe4cc0fa64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:50.591896 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea41e66-bcda-4b2b-89d6-32fe4cc0fa64 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:50.615413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1081/2165] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:50 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:50.619465 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610f8c3-4e13-4bcb-812e-2f2224962726 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:50.630312 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610f8c3-4e13-4bcb-812e-2f2224962726 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:50.637287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d50b1-79a2-4aa8-95c7-11bf05b284b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:50.637981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d50b1-79a2-4aa8-95c7-11bf05b284b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:50.638332 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d50b1-79a2-4aa8-95c7-11bf05b284b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:50.638676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d50b1-79a2-4aa8-95c7-11bf05b284b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:50.639392 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d50b1-79a2-4aa8-95c7-11bf05b284b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:50.641099 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d50b1-79a2-4aa8-95c7-11bf05b284b0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:50.651754 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1085/2166] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:50.655871 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea41e66-bcda-4b2b-89d6-32fe4cc0fa64 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:50.669447 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea41e66-bcda-4b2b-89d6-32fe4cc0fa64 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:50.679306 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9d272-f1b9-4d14-b666-9ad6efd35264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:50.679928 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9d272-f1b9-4d14-b666-9ad6efd35264 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:50.680239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9d272-f1b9-4d14-b666-9ad6efd35264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:50.680507 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9d272-f1b9-4d14-b666-9ad6efd35264 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:50.681219 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9d272-f1b9-4d14-b666-9ad6efd35264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:50.682866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9d272-f1b9-4d14-b666-9ad6efd35264 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:50.698647 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1082/2167] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:50.700297 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d50b1-79a2-4aa8-95c7-11bf05b284b0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:50.706659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:50.714645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d50b1-79a2-4aa8-95c7-11bf05b284b0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:50.725330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf353d85-d3ce-4843-9c15-7a9539239ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:50.726194 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf353d85-d3ce-4843-9c15-7a9539239ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:50.726318 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf353d85-d3ce-4843-9c15-7a9539239ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:50.726592 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf353d85-d3ce-4843-9c15-7a9539239ec5 admin admin] PATH_INFO: `/v3/projects/6522943e031e47a6975d0b062ac43e9b/users/a338c964a33741408ca4b813848126d5/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:50.747323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf353d85-d3ce-4843-9c15-7a9539239ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6522943e031e47a6975d0b062ac43e9b, user_id=a338c964a33741408ca4b813848126d5, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:50.747658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf353d85-d3ce-4843-9c15-7a9539239ec5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:50.772372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1086/2168] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:50 2022] PUT /identity/v3/projects/6522943e031e47a6975d0b062ac43e9b/users/a338c964a33741408ca4b813848126d5/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:50.783131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accea1cf-9f98-47ec-9b94-aff9c738fc7d None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:50.783494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accea1cf-9f98-47ec-9b94-aff9c738fc7d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:50.783814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accea1cf-9f98-47ec-9b94-aff9c738fc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:50.828028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-accea1cf-9f98-47ec-9b94-aff9c738fc7d None None] MFA Rules not processed for user `a338c964a33741408ca4b813848126d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:50.829332 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-accea1cf-9f98-47ec-9b94-aff9c738fc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:50.903253 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:50.907956 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1083/2169] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:56:50.913850 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf353d85-d3ce-4843-9c15-7a9539239ec5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:50.987855 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf353d85-d3ce-4843-9c15-7a9539239ec5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:50.993720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c738a43-91bf-43f0-980b-98334fd589c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:50.994335 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c738a43-91bf-43f0-980b-98334fd589c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:50.994617 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c738a43-91bf-43f0-980b-98334fd589c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:50.994893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c738a43-91bf-43f0-980b-98334fd589c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:50.995530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c738a43-91bf-43f0-980b-98334fd589c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:50.997480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c738a43-91bf-43f0-980b-98334fd589c8 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:51.038477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1087/2170] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:50 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:56:51.039674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accea1cf-9f98-47ec-9b94-aff9c738fc7d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:51.051333 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accea1cf-9f98-47ec-9b94-aff9c738fc7d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:51.115027 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2da865-94b4-4d4e-850f-3f29e1baf86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:51.115561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2da865-94b4-4d4e-850f-3f29e1baf86e admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:51.115836 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2da865-94b4-4d4e-850f-3f29e1baf86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:51.116203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2da865-94b4-4d4e-850f-3f29e1baf86e admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:51.116844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2da865-94b4-4d4e-850f-3f29e1baf86e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:51.118803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2da865-94b4-4d4e-850f-3f29e1baf86e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:51.148438 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1084/2171] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:51 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:51.152482 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c738a43-91bf-43f0-980b-98334fd589c8 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:51.176662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c738a43-91bf-43f0-980b-98334fd589c8 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:51.205488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04eab75-8c68-4d28-be66-93c0bd464aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:51.210704 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04eab75-8c68-4d28-be66-93c0bd464aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:51.210704 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04eab75-8c68-4d28-be66-93c0bd464aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:51.210960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04eab75-8c68-4d28-be66-93c0bd464aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:51.212360 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04eab75-8c68-4d28-be66-93c0bd464aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:51.226270 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04eab75-8c68-4d28-be66-93c0bd464aa2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:51.242171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1088/2172] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:51.246181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2da865-94b4-4d4e-850f-3f29e1baf86e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:51.270766 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2da865-94b4-4d4e-850f-3f29e1baf86e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:51.282004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124b7ff-6c9c-4f74-a620-bea27bfad07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:51.282004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124b7ff-6c9c-4f74-a620-bea27bfad07f admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:51.282004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124b7ff-6c9c-4f74-a620-bea27bfad07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:51.282004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124b7ff-6c9c-4f74-a620-bea27bfad07f admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:51.283903 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124b7ff-6c9c-4f74-a620-bea27bfad07f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:51.286540 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124b7ff-6c9c-4f74-a620-bea27bfad07f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:51.302958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1085/2173] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:56:51.307781 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04eab75-8c68-4d28-be66-93c0bd464aa2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:51.319140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04eab75-8c68-4d28-be66-93c0bd464aa2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:51.324695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f52e6-6e73-42e5-b7b4-3b661e2e39fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:51.325756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f52e6-6e73-42e5-b7b4-3b661e2e39fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:51.326249 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f52e6-6e73-42e5-b7b4-3b661e2e39fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:51.326619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f52e6-6e73-42e5-b7b4-3b661e2e39fe admin admin] PATH_INFO: `/v3/projects/e887cf6e540f4bd4a5b22d1aee759a24/users/0c232c70b2ec40ef871c7227ca7c5cd5/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:51.371716 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f52e6-6e73-42e5-b7b4-3b661e2e39fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e887cf6e540f4bd4a5b22d1aee759a24, user_id=0c232c70b2ec40ef871c7227ca7c5cd5, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:51.382879 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f52e6-6e73-42e5-b7b4-3b661e2e39fe admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:51.410474 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1089/2174] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:51 2022] PUT /identity/v3/projects/e887cf6e540f4bd4a5b22d1aee759a24/users/0c232c70b2ec40ef871c7227ca7c5cd5/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:56:51.426095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa95d3a-0b0a-4159-bb44-bf0c505c8c32 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:51.426385 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa95d3a-0b0a-4159-bb44-bf0c505c8c32 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:51.426818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa95d3a-0b0a-4159-bb44-bf0c505c8c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:51.472230 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-afa95d3a-0b0a-4159-bb44-bf0c505c8c32 None None] MFA Rules not processed for user `0c232c70b2ec40ef871c7227ca7c5cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:51.475712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-afa95d3a-0b0a-4159-bb44-bf0c505c8c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:51.597613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1086/2175] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:51.600214 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f52e6-6e73-42e5-b7b4-3b661e2e39fe admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:51.714303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f52e6-6e73-42e5-b7b4-3b661e2e39fe admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:51.723572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766c385-b833-46d7-b377-65ed557775ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:51.725381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766c385-b833-46d7-b377-65ed557775ff admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:51.725886 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766c385-b833-46d7-b377-65ed557775ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:51.726299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766c385-b833-46d7-b377-65ed557775ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:51.727292 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766c385-b833-46d7-b377-65ed557775ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:51.729757 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766c385-b833-46d7-b377-65ed557775ff admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:51.766042 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1090/2176] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:51 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:56:51.770395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa95d3a-0b0a-4159-bb44-bf0c505c8c32 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:51.783627 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa95d3a-0b0a-4159-bb44-bf0c505c8c32 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:51.897338 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7b32b-2583-492a-bbb9-1b3e8460e8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:51.898076 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7b32b-2583-492a-bbb9-1b3e8460e8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:51.898455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7b32b-2583-492a-bbb9-1b3e8460e8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:51.899196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7b32b-2583-492a-bbb9-1b3e8460e8a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:51.902076 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7b32b-2583-492a-bbb9-1b3e8460e8a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:51.906738 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7b32b-2583-492a-bbb9-1b3e8460e8a0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:51.937327 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:51.949077 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1087/2177] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:51 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:51.953022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766c385-b833-46d7-b377-65ed557775ff admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:51.970997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766c385-b833-46d7-b377-65ed557775ff admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:51.992205 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecfd73-1441-4bf0-9b85-d90a42c46514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:51.992836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecfd73-1441-4bf0-9b85-d90a42c46514 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:51.993118 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecfd73-1441-4bf0-9b85-d90a42c46514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:51.993394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecfd73-1441-4bf0-9b85-d90a42c46514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:51.994041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecfd73-1441-4bf0-9b85-d90a42c46514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:51.998750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecfd73-1441-4bf0-9b85-d90a42c46514 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:52.039691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1091/2178] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:52.040416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7b32b-2583-492a-bbb9-1b3e8460e8a0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:52.073635 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7b32b-2583-492a-bbb9-1b3e8460e8a0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:52.094784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f013f4e-a661-434f-b0b7-5d44aff6758c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:52.095642 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f013f4e-a661-434f-b0b7-5d44aff6758c admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:52.095924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f013f4e-a661-434f-b0b7-5d44aff6758c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:52.096220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f013f4e-a661-434f-b0b7-5d44aff6758c admin admin] PATH_INFO: `/v3/projects/734da144219a4a40bd6e058d5ca4669c/users/33d34d6b2c1a49879196fd8fe07c2476/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:52.125672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f013f4e-a661-434f-b0b7-5d44aff6758c admin admin] RBAC: Authorizing `identity:create_grant(project_id=734da144219a4a40bd6e058d5ca4669c, user_id=33d34d6b2c1a49879196fd8fe07c2476, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:52.128747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f013f4e-a661-434f-b0b7-5d44aff6758c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:52.152244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1088/2179] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:52 2022] PUT /identity/v3/projects/734da144219a4a40bd6e058d5ca4669c/users/33d34d6b2c1a49879196fd8fe07c2476/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:56:52.160979 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecfd73-1441-4bf0-9b85-d90a42c46514 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:52.166618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:52.180807 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecfd73-1441-4bf0-9b85-d90a42c46514 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:52.332614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45979bb-75cc-4f1d-9ea8-9902b52c3d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:52.333249 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45979bb-75cc-4f1d-9ea8-9902b52c3d92 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:52.334057 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45979bb-75cc-4f1d-9ea8-9902b52c3d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:52.334339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45979bb-75cc-4f1d-9ea8-9902b52c3d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:52.334997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45979bb-75cc-4f1d-9ea8-9902b52c3d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:52.336940 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45979bb-75cc-4f1d-9ea8-9902b52c3d92 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:52.355438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1092/2180] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:52.359577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f013f4e-a661-434f-b0b7-5d44aff6758c admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:52.381639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f013f4e-a661-434f-b0b7-5d44aff6758c admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:52.396758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530debf-dfe8-41f3-a29b-64ab94788a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:52.404375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530debf-dfe8-41f3-a29b-64ab94788a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:52.404653 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530debf-dfe8-41f3-a29b-64ab94788a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:52.404921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530debf-dfe8-41f3-a29b-64ab94788a21 admin admin] PATH_INFO: `/v3/projects/734da144219a4a40bd6e058d5ca4669c/users/33d34d6b2c1a49879196fd8fe07c2476/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:52.407409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8530debf-dfe8-41f3-a29b-64ab94788a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=734da144219a4a40bd6e058d5ca4669c, user_id=33d34d6b2c1a49879196fd8fe07c2476, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:52.413376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8530debf-dfe8-41f3-a29b-64ab94788a21 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:52.438346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1089/2181] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:52 2022] PUT /identity/v3/projects/734da144219a4a40bd6e058d5ca4669c/users/33d34d6b2c1a49879196fd8fe07c2476/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:52.450389 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a623a91-e444-4396-988f-29766ee019ce None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:52.450585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a623a91-e444-4396-988f-29766ee019ce None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:52.450860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a623a91-e444-4396-988f-29766ee019ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:52.504470 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1a623a91-e444-4396-988f-29766ee019ce None None] MFA Rules not processed for user `33d34d6b2c1a49879196fd8fe07c2476`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:52.506741 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1a623a91-e444-4396-988f-29766ee019ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:52.616697 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1093/2182] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:52.623616 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530debf-dfe8-41f3-a29b-64ab94788a21 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:52.625525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8530debf-dfe8-41f3-a29b-64ab94788a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:52.700621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530debf-dfe8-41f3-a29b-64ab94788a21 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:52.705088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04250d8a-3c42-4f3d-93df-2c966086182c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:52.705841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04250d8a-3c42-4f3d-93df-2c966086182c service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:52.706279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04250d8a-3c42-4f3d-93df-2c966086182c service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:52.706416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04250d8a-3c42-4f3d-93df-2c966086182c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:52.787220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04250d8a-3c42-4f3d-93df-2c966086182c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:52.789166 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04250d8a-3c42-4f3d-93df-2c966086182c service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:52.818473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1090/2183] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:52.894074 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f42141-a922-484e-8e06-2e0d65fb6a35 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:52.894362 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f42141-a922-484e-8e06-2e0d65fb6a35 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:52.894641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f42141-a922-484e-8e06-2e0d65fb6a35 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:52.905108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1094/2184] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 7 07:56:52 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:56:52.918687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2a749-0b09-469f-877c-ba352d5a4349 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:52.918971 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2a749-0b09-469f-877c-ba352d5a4349 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:52.919244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2a749-0b09-469f-877c-ba352d5a4349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:52.954714 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-99a2a749-0b09-469f-877c-ba352d5a4349 None None] MFA Rules not processed for user `b16b81ab30294bc4a8c87b6d7c3573f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:52.955948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-99a2a749-0b09-469f-877c-ba352d5a4349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:53.056200 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f42141-a922-484e-8e06-2e0d65fb6a35 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:53.058174 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-68f42141-a922-484e-8e06-2e0d65fb6a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:53.085496 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1091/2185] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 7 07:56:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:53.144395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2a749-0b09-469f-877c-ba352d5a4349 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:53.147503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-99a2a749-0b09-469f-877c-ba352d5a4349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:53.195665 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2a749-0b09-469f-877c-ba352d5a4349 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:53.215114 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc305e-410f-48d2-9640-c4ce10ac5480 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:53.216114 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc305e-410f-48d2-9640-c4ce10ac5480 service placement] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:53.216541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc305e-410f-48d2-9640-c4ce10ac5480 service placement] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:53.216977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc305e-410f-48d2-9640-c4ce10ac5480 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:53.233525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f42141-a922-484e-8e06-2e0d65fb6a35 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:53.244980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc305e-410f-48d2-9640-c4ce10ac5480 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:53.246039 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0403e-8484-4605-aedb-23208d68ee59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:53.246683 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0403e-8484-4605-aedb-23208d68ee59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:53.246959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0403e-8484-4605-aedb-23208d68ee59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:53.247402 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0403e-8484-4605-aedb-23208d68ee59 admin admin] PATH_INFO: `/v3/users/4035909c0f414e5ca7fbab2caee29f4d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:53.248599 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0403e-8484-4605-aedb-23208d68ee59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4035909c0f414e5ca7fbab2caee29f4d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:53.252507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0403e-8484-4605-aedb-23208d68ee59 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:53.262083 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc305e-410f-48d2-9640-c4ce10ac5480 service placement] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:53.301243 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:53.326437 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1092/2186] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:53.353841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:53.388530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-87b0403e-8484-4605-aedb-23208d68ee59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4035909c0f414e5ca7fbab2caee29f4d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:53.402968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-87b0403e-8484-4605-aedb-23208d68ee59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4035909c0f414e5ca7fbab2caee29f4d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:53.425667 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-87b0403e-8484-4605-aedb-23208d68ee59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4035909c0f414e5ca7fbab2caee29f4d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:53.426520 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-87b0403e-8484-4605-aedb-23208d68ee59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4035909c0f414e5ca7fbab2caee29f4d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:53.438681 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-87b0403e-8484-4605-aedb-23208d68ee59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4035909c0f414e5ca7fbab2caee29f4d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:53.447519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1095/2187] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:56:53 2022] DELETE /identity/v3/users/4035909c0f414e5ca7fbab2caee29f4d => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:56:53.620093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc305e-410f-48d2-9640-c4ce10ac5480 service placement] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:53.635104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc305e-410f-48d2-9640-c4ce10ac5480 service placement] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:53.695016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035184d-bedb-49f3-947e-bebf96132c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:53.695648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035184d-bedb-49f3-947e-bebf96132c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:53.695927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035184d-bedb-49f3-947e-bebf96132c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:53.696239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035184d-bedb-49f3-947e-bebf96132c63 admin admin] PATH_INFO: `/v3/projects/b093ac8c5cbc4bb6be326b2fb993a76b` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:53.697381 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035184d-bedb-49f3-947e-bebf96132c63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b093ac8c5cbc4bb6be326b2fb993a76b)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:53.699288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035184d-bedb-49f3-947e-bebf96132c63 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:53.745380 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3035184d-bedb-49f3-947e-bebf96132c63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b093ac8c5cbc4bb6be326b2fb993a76b', 'request_id': 'req-3035184d-bedb-49f3-947e-bebf96132c63'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:53.750810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3035184d-bedb-49f3-947e-bebf96132c63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b093ac8c5cbc4bb6be326b2fb993a76b', 'request_id': 'req-3035184d-bedb-49f3-947e-bebf96132c63'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:53.756858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3035184d-bedb-49f3-947e-bebf96132c63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b093ac8c5cbc4bb6be326b2fb993a76b', 'request_id': 'req-3035184d-bedb-49f3-947e-bebf96132c63'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:53.768672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3035184d-bedb-49f3-947e-bebf96132c63 admin admin] The token cache is being invalidate because project b093ac8c5cbc4bb6be326b2fb993a76b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:56:53.769051 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3035184d-bedb-49f3-947e-bebf96132c63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:53.769689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3035184d-bedb-49f3-947e-bebf96132c63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:56:53.771592 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1093/2188] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:56:53 2022] DELETE /identity/v3/projects/b093ac8c5cbc4bb6be326b2fb993a76b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:53.777554 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d43e8b-1c75-4084-a844-0ea5b397b731 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:53.777840 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d43e8b-1c75-4084-a844-0ea5b397b731 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:53.778120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d43e8b-1c75-4084-a844-0ea5b397b731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:53.799349 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-39d43e8b-1c75-4084-a844-0ea5b397b731 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:53.800604 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-39d43e8b-1c75-4084-a844-0ea5b397b731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:53.862406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1096/2189] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:53.870704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27167973-c610-44ea-847c-c32961512778 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:53.870991 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27167973-c610-44ea-847c-c32961512778 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:53.871274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27167973-c610-44ea-847c-c32961512778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:53.891628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-27167973-c610-44ea-847c-c32961512778 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:53.892877 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-27167973-c610-44ea-847c-c32961512778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:53.901034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1094/2190] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:53.906944 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d43e8b-1c75-4084-a844-0ea5b397b731 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:53.917874 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d43e8b-1c75-4084-a844-0ea5b397b731 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:53.922635 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b3e5f-7ca9-4f0d-bc71-ec992254ffca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:53.923236 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b3e5f-7ca9-4f0d-bc71-ec992254ffca admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:53.923512 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b3e5f-7ca9-4f0d-bc71-ec992254ffca admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:53.923785 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b3e5f-7ca9-4f0d-bc71-ec992254ffca admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:53.924613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b3e5f-7ca9-4f0d-bc71-ec992254ffca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:56:53.924924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b3e5f-7ca9-4f0d-bc71-ec992254ffca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:53.926508 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b3e5f-7ca9-4f0d-bc71-ec992254ffca admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:53.939147 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1097/2191] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:56:53.943952 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27167973-c610-44ea-847c-c32961512778 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:53.954303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27167973-c610-44ea-847c-c32961512778 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:53.959448 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ee1f0-4bfe-428c-805b-bad53935f999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:53.960181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ee1f0-4bfe-428c-805b-bad53935f999 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:53.960515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ee1f0-4bfe-428c-805b-bad53935f999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:53.960887 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ee1f0-4bfe-428c-805b-bad53935f999 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:53.961757 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ee1f0-4bfe-428c-805b-bad53935f999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:53.964007 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ee1f0-4bfe-428c-805b-bad53935f999 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:54.000130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1095/2192] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:56:53 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:54.003315 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b3e5f-7ca9-4f0d-bc71-ec992254ffca admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:54.014921 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b3e5f-7ca9-4f0d-bc71-ec992254ffca admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:54.098068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32102029-b585-4e89-a67d-646f253e61a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:54.098714 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32102029-b585-4e89-a67d-646f253e61a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:54.099211 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32102029-b585-4e89-a67d-646f253e61a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:54.099597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32102029-b585-4e89-a67d-646f253e61a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:54.100393 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32102029-b585-4e89-a67d-646f253e61a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:54.102390 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32102029-b585-4e89-a67d-646f253e61a6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:54.125920 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1098/2193] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:56:54 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:56:54.131050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ee1f0-4bfe-428c-805b-bad53935f999 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:54.143501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ee1f0-4bfe-428c-805b-bad53935f999 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:54.149001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64bd09-4cc2-4e63-b5a0-0c828c99ab1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:54.150321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64bd09-4cc2-4e63-b5a0-0c828c99ab1e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:54.150321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64bd09-4cc2-4e63-b5a0-0c828c99ab1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:54.150321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64bd09-4cc2-4e63-b5a0-0c828c99ab1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:54.150753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64bd09-4cc2-4e63-b5a0-0c828c99ab1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:54.152376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64bd09-4cc2-4e63-b5a0-0c828c99ab1e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:54.164167 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1096/2194] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:54.167982 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32102029-b585-4e89-a67d-646f253e61a6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:54.179203 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32102029-b585-4e89-a67d-646f253e61a6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:54.186582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15cfba5-78df-404c-b83c-d7cf1aa338b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:54.187334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cfba5-78df-404c-b83c-d7cf1aa338b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:54.187609 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cfba5-78df-404c-b83c-d7cf1aa338b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:54.187875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cfba5-78df-404c-b83c-d7cf1aa338b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:54.188577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15cfba5-78df-404c-b83c-d7cf1aa338b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:54.190157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15cfba5-78df-404c-b83c-d7cf1aa338b6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:54.202501 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1099/2195] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:56:54.206675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64bd09-4cc2-4e63-b5a0-0c828c99ab1e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:54.216889 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64bd09-4cc2-4e63-b5a0-0c828c99ab1e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:54.221458 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ba840-56b4-4875-924c-6c4e83b5f76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:54.222104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ba840-56b4-4875-924c-6c4e83b5f76f admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:54.222380 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ba840-56b4-4875-924c-6c4e83b5f76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:54.222649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ba840-56b4-4875-924c-6c4e83b5f76f admin admin] PATH_INFO: `/v3/projects/3264396aada44ba89f39cfe0776e183e/users/55b97e954da94f909f8f5ffff515509e/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:54.237781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ba840-56b4-4875-924c-6c4e83b5f76f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3264396aada44ba89f39cfe0776e183e, user_id=55b97e954da94f909f8f5ffff515509e, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:54.241033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ba840-56b4-4875-924c-6c4e83b5f76f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:54.255994 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1097/2196] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:56:54 2022] PUT /identity/v3/projects/3264396aada44ba89f39cfe0776e183e/users/55b97e954da94f909f8f5ffff515509e/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:56:54.261018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba4841-c952-4345-8eb1-0d69590c81fe None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:54.261366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba4841-c952-4345-8eb1-0d69590c81fe None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:54.261642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba4841-c952-4345-8eb1-0d69590c81fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:54.282202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3eba4841-c952-4345-8eb1-0d69590c81fe None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:54.283436 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3eba4841-c952-4345-8eb1-0d69590c81fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:54.297950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:54.342838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2637f7d-3432-4f3e-a061-50b88aa9546a None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:54.343147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2637f7d-3432-4f3e-a061-50b88aa9546a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:54.343389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2637f7d-3432-4f3e-a061-50b88aa9546a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:54.367707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e2637f7d-3432-4f3e-a061-50b88aa9546a None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:56:54.368949 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e2637f7d-3432-4f3e-a061-50b88aa9546a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:54.394660 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:56:54.416259 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1100/2197] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:56:54.416259 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba4841-c952-4345-8eb1-0d69590c81fe None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:54.416259 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3eba4841-c952-4345-8eb1-0d69590c81fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:54.456138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1098/2198] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Sep 7 07:56:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:56:54.483619 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2637f7d-3432-4f3e-a061-50b88aa9546a None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:54.485772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e2637f7d-3432-4f3e-a061-50b88aa9546a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:54.527715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba4841-c952-4345-8eb1-0d69590c81fe None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:54.538860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f08f77-2890-4885-a031-f7eea59b5662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:54.539491 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f08f77-2890-4885-a031-f7eea59b5662 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:54.539764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f08f77-2890-4885-a031-f7eea59b5662 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:54.540033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f08f77-2890-4885-a031-f7eea59b5662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:54.557194 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f08f77-2890-4885-a031-f7eea59b5662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:54.559173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f08f77-2890-4885-a031-f7eea59b5662 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:54.593037 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1101/2199] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:54.637313 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2637f7d-3432-4f3e-a061-50b88aa9546a None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:54.663142 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215219a7-9052-49b5-9f29-933a530e6191 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:54.663815 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215219a7-9052-49b5-9f29-933a530e6191 service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:54.664157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215219a7-9052-49b5-9f29-933a530e6191 service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:54.664434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215219a7-9052-49b5-9f29-933a530e6191 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:54.694605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215219a7-9052-49b5-9f29-933a530e6191 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:54.696322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215219a7-9052-49b5-9f29-933a530e6191 service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:54.706298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1099/2200] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 07:56:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 07:56:54.758496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f08f77-2890-4885-a031-f7eea59b5662 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:54.760535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-56f08f77-2890-4885-a031-f7eea59b5662 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:54.895178 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f08f77-2890-4885-a031-f7eea59b5662 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:54.905971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d1dae-ccfc-4ca4-bbe2-87172ef8ce57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:54.906657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d1dae-ccfc-4ca4-bbe2-87172ef8ce57 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:54.906929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d1dae-ccfc-4ca4-bbe2-87172ef8ce57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:54.907204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d1dae-ccfc-4ca4-bbe2-87172ef8ce57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:54.911382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b16d1dae-ccfc-4ca4-bbe2-87172ef8ce57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:55.012167 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d1dae-ccfc-4ca4-bbe2-87172ef8ce57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:55.014089 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d1dae-ccfc-4ca4-bbe2-87172ef8ce57 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:55.045159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1102/2201] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:56:56.231706 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215219a7-9052-49b5-9f29-933a530e6191 service swift] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:56:56.233354 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-215219a7-9052-49b5-9f29-933a530e6191 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:56.251030 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215219a7-9052-49b5-9f29-933a530e6191 service swift] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:56:56.263703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c00445-0b19-4573-b6a6-dc8de4624b1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:56:56.264479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c00445-0b19-4573-b6a6-dc8de4624b1c service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:56:56.264868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c00445-0b19-4573-b6a6-dc8de4624b1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:56:56.265272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c00445-0b19-4573-b6a6-dc8de4624b1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:56:56.267639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a2c00445-0b19-4573-b6a6-dc8de4624b1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:56:56.398267 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c00445-0b19-4573-b6a6-dc8de4624b1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:56:56.400164 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c00445-0b19-4573-b6a6-dc8de4624b1c service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:56:56.419146 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1100/2202] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:56:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:57:01.553849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d1dae-ccfc-4ca4-bbe2-87172ef8ce57 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:01.555314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b16d1dae-ccfc-4ca4-bbe2-87172ef8ce57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:01.584575 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d1dae-ccfc-4ca4-bbe2-87172ef8ce57 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:01.591260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54363473-3302-4330-b889-74e4dc073ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:01.592030 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54363473-3302-4330-b889-74e4dc073ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:01.592463 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54363473-3302-4330-b889-74e4dc073ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:01.592723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54363473-3302-4330-b889-74e4dc073ec0 admin admin] PATH_INFO: `/v3/users/a338c964a33741408ca4b813848126d5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:01.594295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54363473-3302-4330-b889-74e4dc073ec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a338c964a33741408ca4b813848126d5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:01.596493 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54363473-3302-4330-b889-74e4dc073ec0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:01.598279 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:01.700606 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-54363473-3302-4330-b889-74e4dc073ec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a338c964a33741408ca4b813848126d5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:01.731154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-54363473-3302-4330-b889-74e4dc073ec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a338c964a33741408ca4b813848126d5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:01.746376 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-54363473-3302-4330-b889-74e4dc073ec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a338c964a33741408ca4b813848126d5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:01.747022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-54363473-3302-4330-b889-74e4dc073ec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a338c964a33741408ca4b813848126d5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:01.760664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-54363473-3302-4330-b889-74e4dc073ec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a338c964a33741408ca4b813848126d5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:01.777689 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1103/2203] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:57:01 2022] DELETE /identity/v3/users/a338c964a33741408ca4b813848126d5 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:57:01.781992 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d2494-764d-4f01-8651-ea14cc4e06fc None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:01.782541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d2494-764d-4f01-8651-ea14cc4e06fc None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:01.782541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d2494-764d-4f01-8651-ea14cc4e06fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:01.826632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ea9d2494-764d-4f01-8651-ea14cc4e06fc None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:01.828326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ea9d2494-764d-4f01-8651-ea14cc4e06fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:01.949720 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:02.001130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1101/2204] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:57:02.042676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54363473-3302-4330-b889-74e4dc073ec0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:02.044014 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-54363473-3302-4330-b889-74e4dc073ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:02.246006 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54363473-3302-4330-b889-74e4dc073ec0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:02.263866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cea9a-d103-4d4d-8cf8-dbe3a7ec7177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:02.269115 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cea9a-d103-4d4d-8cf8-dbe3a7ec7177 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:02.269725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cea9a-d103-4d4d-8cf8-dbe3a7ec7177 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:02.270242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cea9a-d103-4d4d-8cf8-dbe3a7ec7177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:02.301971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cea9a-d103-4d4d-8cf8-dbe3a7ec7177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:02.304143 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cea9a-d103-4d4d-8cf8-dbe3a7ec7177 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:02.336045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1104/2205] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:57:03.300120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d2494-764d-4f01-8651-ea14cc4e06fc None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:03.302313 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ea9d2494-764d-4f01-8651-ea14cc4e06fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:03.340019 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d2494-764d-4f01-8651-ea14cc4e06fc None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:03.371376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb462f1d-529d-47ec-a7d6-02cdd58076e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:03.373134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb462f1d-529d-47ec-a7d6-02cdd58076e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:03.374798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb462f1d-529d-47ec-a7d6-02cdd58076e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:03.375185 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb462f1d-529d-47ec-a7d6-02cdd58076e1 admin admin] PATH_INFO: `/v3/projects/6522943e031e47a6975d0b062ac43e9b` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:03.387677 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb462f1d-529d-47ec-a7d6-02cdd58076e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6522943e031e47a6975d0b062ac43e9b)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:03.397564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb462f1d-529d-47ec-a7d6-02cdd58076e1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:03.559696 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bb462f1d-529d-47ec-a7d6-02cdd58076e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6522943e031e47a6975d0b062ac43e9b', 'request_id': 'req-bb462f1d-529d-47ec-a7d6-02cdd58076e1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:03.583248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bb462f1d-529d-47ec-a7d6-02cdd58076e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6522943e031e47a6975d0b062ac43e9b', 'request_id': 'req-bb462f1d-529d-47ec-a7d6-02cdd58076e1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:03.615262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bb462f1d-529d-47ec-a7d6-02cdd58076e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6522943e031e47a6975d0b062ac43e9b', 'request_id': 'req-bb462f1d-529d-47ec-a7d6-02cdd58076e1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:03.651635 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bb462f1d-529d-47ec-a7d6-02cdd58076e1 admin admin] The token cache is being invalidate because project 6522943e031e47a6975d0b062ac43e9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:57:03.651964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bb462f1d-529d-47ec-a7d6-02cdd58076e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:03.652656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bb462f1d-529d-47ec-a7d6-02cdd58076e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:03.654835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1102/2206] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:57:03 2022] DELETE /identity/v3/projects/6522943e031e47a6975d0b062ac43e9b => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:57:03.658669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cea9a-d103-4d4d-8cf8-dbe3a7ec7177 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:03.672099 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cea9a-d103-4d4d-8cf8-dbe3a7ec7177 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:03.843888 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484519b-f89e-498d-9c6c-af25fe3120e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:03.845260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484519b-f89e-498d-9c6c-af25fe3120e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:03.845680 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484519b-f89e-498d-9c6c-af25fe3120e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:03.846134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484519b-f89e-498d-9c6c-af25fe3120e1 admin admin] PATH_INFO: `/v3/users/0c232c70b2ec40ef871c7227ca7c5cd5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:03.849039 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f484519b-f89e-498d-9c6c-af25fe3120e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c232c70b2ec40ef871c7227ca7c5cd5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:03.852841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f484519b-f89e-498d-9c6c-af25fe3120e1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:03.897290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:03.993818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f484519b-f89e-498d-9c6c-af25fe3120e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c232c70b2ec40ef871c7227ca7c5cd5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:04.012369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f484519b-f89e-498d-9c6c-af25fe3120e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c232c70b2ec40ef871c7227ca7c5cd5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:04.025736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f484519b-f89e-498d-9c6c-af25fe3120e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c232c70b2ec40ef871c7227ca7c5cd5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:04.028310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f484519b-f89e-498d-9c6c-af25fe3120e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c232c70b2ec40ef871c7227ca7c5cd5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:04.049131 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f484519b-f89e-498d-9c6c-af25fe3120e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c232c70b2ec40ef871c7227ca7c5cd5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:04.066994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1105/2207] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:57:03 2022] DELETE /identity/v3/users/0c232c70b2ec40ef871c7227ca7c5cd5 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:57:04.101417 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9c746-dab6-4f2a-8fc7-4eecd79f1722 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:04.102069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9c746-dab6-4f2a-8fc7-4eecd79f1722 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:04.102526 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9c746-dab6-4f2a-8fc7-4eecd79f1722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:04.171685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-4cd9c746-dab6-4f2a-8fc7-4eecd79f1722 None None] MFA Rules not processed for user `55b97e954da94f909f8f5ffff515509e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:04.173070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4cd9c746-dab6-4f2a-8fc7-4eecd79f1722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:04.272151 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1103/2208] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:57:04.287474 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484519b-f89e-498d-9c6c-af25fe3120e1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:04.288962 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f484519b-f89e-498d-9c6c-af25fe3120e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:04.397469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484519b-f89e-498d-9c6c-af25fe3120e1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:04.405102 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c0825-4a99-4d4f-9c1e-8ed420a39557 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:04.405726 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c0825-4a99-4d4f-9c1e-8ed420a39557 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:04.406002 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c0825-4a99-4d4f-9c1e-8ed420a39557 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:04.406301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c0825-4a99-4d4f-9c1e-8ed420a39557 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:04.415640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c0825-4a99-4d4f-9c1e-8ed420a39557 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:04.417415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c0825-4a99-4d4f-9c1e-8ed420a39557 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:04.429480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1106/2209] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:57:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:57:04.549228 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9c746-dab6-4f2a-8fc7-4eecd79f1722 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:04.551305 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4cd9c746-dab6-4f2a-8fc7-4eecd79f1722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:04.639462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9c746-dab6-4f2a-8fc7-4eecd79f1722 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:04.646505 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cedff7-b181-4b15-884b-6f15292e76b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:04.647808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cedff7-b181-4b15-884b-6f15292e76b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:04.648198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cedff7-b181-4b15-884b-6f15292e76b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:04.648564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cedff7-b181-4b15-884b-6f15292e76b5 admin admin] PATH_INFO: `/v3/projects/e887cf6e540f4bd4a5b22d1aee759a24` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:04.660714 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cedff7-b181-4b15-884b-6f15292e76b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e887cf6e540f4bd4a5b22d1aee759a24)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:04.664461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cedff7-b181-4b15-884b-6f15292e76b5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:04.675013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:04.686135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c0825-4a99-4d4f-9c1e-8ed420a39557 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:04.688664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-080c0825-4a99-4d4f-9c1e-8ed420a39557 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:04.727994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c0825-4a99-4d4f-9c1e-8ed420a39557 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:04.745057 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ec93c-af02-460f-b1b1-edfa06870b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:04.745750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ec93c-af02-460f-b1b1-edfa06870b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:04.746703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ec93c-af02-460f-b1b1-edfa06870b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:04.748205 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ec93c-af02-460f-b1b1-edfa06870b16 admin admin] PATH_INFO: `/v3/users/30d69b9ce0214049a029489b11e719b5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:04.749947 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ec93c-af02-460f-b1b1-edfa06870b16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30d69b9ce0214049a029489b11e719b5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:04.756316 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ec93c-af02-460f-b1b1-edfa06870b16 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:04.758265 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-24cedff7-b181-4b15-884b-6f15292e76b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e887cf6e540f4bd4a5b22d1aee759a24', 'request_id': 'req-24cedff7-b181-4b15-884b-6f15292e76b5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:04.766225 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-24cedff7-b181-4b15-884b-6f15292e76b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e887cf6e540f4bd4a5b22d1aee759a24', 'request_id': 'req-24cedff7-b181-4b15-884b-6f15292e76b5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:04.779467 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-24cedff7-b181-4b15-884b-6f15292e76b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e887cf6e540f4bd4a5b22d1aee759a24', 'request_id': 'req-24cedff7-b181-4b15-884b-6f15292e76b5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:04.853165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-24cedff7-b181-4b15-884b-6f15292e76b5 admin admin] The token cache is being invalidate because project e887cf6e540f4bd4a5b22d1aee759a24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:57:04.853413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-24cedff7-b181-4b15-884b-6f15292e76b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:04.855022 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-24cedff7-b181-4b15-884b-6f15292e76b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:04.860790 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1104/2210] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:57:04 2022] DELETE /identity/v3/projects/e887cf6e540f4bd4a5b22d1aee759a24 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:57:04.868126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cedff7-b181-4b15-884b-6f15292e76b5 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:04.875275 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-24cedff7-b181-4b15-884b-6f15292e76b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:04.911087 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d69ec93c-af02-460f-b1b1-edfa06870b16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30d69b9ce0214049a029489b11e719b5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:04.952524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d69ec93c-af02-460f-b1b1-edfa06870b16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30d69b9ce0214049a029489b11e719b5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:04.963369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d69ec93c-af02-460f-b1b1-edfa06870b16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30d69b9ce0214049a029489b11e719b5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:04.965362 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d69ec93c-af02-460f-b1b1-edfa06870b16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30d69b9ce0214049a029489b11e719b5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:04.985493 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d69ec93c-af02-460f-b1b1-edfa06870b16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30d69b9ce0214049a029489b11e719b5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:04.992150 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:05.008723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1107/2211] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:57:04 2022] DELETE /identity/v3/users/30d69b9ce0214049a029489b11e719b5 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:57:05.013343 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0ab80-0821-4b99-865b-b75e42bdb9b6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:05.014801 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0ab80-0821-4b99-865b-b75e42bdb9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:05.015497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0ab80-0821-4b99-865b-b75e42bdb9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:05.062978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5fc0ab80-0821-4b99-865b-b75e42bdb9b6 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:05.065038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5fc0ab80-0821-4b99-865b-b75e42bdb9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:05.099065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cedff7-b181-4b15-884b-6f15292e76b5 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:05.218708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1108/2212] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:57:05.229607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0ab80-0821-4b99-865b-b75e42bdb9b6 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:05.232023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5fc0ab80-0821-4b99-865b-b75e42bdb9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:05.261631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5624261-2a6c-47eb-8c68-2607160675d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:05.262282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5624261-2a6c-47eb-8c68-2607160675d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:05.262621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5624261-2a6c-47eb-8c68-2607160675d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:05.262834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5624261-2a6c-47eb-8c68-2607160675d4 admin admin] PATH_INFO: `/v3/users/33d34d6b2c1a49879196fd8fe07c2476` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:05.268734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5624261-2a6c-47eb-8c68-2607160675d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33d34d6b2c1a49879196fd8fe07c2476)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:05.271317 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5624261-2a6c-47eb-8c68-2607160675d4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:05.384018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0ab80-0821-4b99-865b-b75e42bdb9b6 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:05.394050 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c83e1-d4db-4e09-9ac2-40ced8b71c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:05.394682 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c83e1-d4db-4e09-9ac2-40ced8b71c3b service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:05.394963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c83e1-d4db-4e09-9ac2-40ced8b71c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:05.395238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c83e1-d4db-4e09-9ac2-40ced8b71c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:05.424034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:05.429887 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c5624261-2a6c-47eb-8c68-2607160675d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33d34d6b2c1a49879196fd8fe07c2476'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:05.431503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c5624261-2a6c-47eb-8c68-2607160675d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33d34d6b2c1a49879196fd8fe07c2476'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:05.444890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c83e1-d4db-4e09-9ac2-40ced8b71c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:05.448307 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c83e1-d4db-4e09-9ac2-40ced8b71c3b service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:05.463996 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c5624261-2a6c-47eb-8c68-2607160675d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33d34d6b2c1a49879196fd8fe07c2476'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:05.464903 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c5624261-2a6c-47eb-8c68-2607160675d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33d34d6b2c1a49879196fd8fe07c2476'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:05.472993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1109/2213] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:57:05.486913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c5624261-2a6c-47eb-8c68-2607160675d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33d34d6b2c1a49879196fd8fe07c2476'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:05.518364 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1105/2214] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:57:04 2022] DELETE /identity/v3/users/33d34d6b2c1a49879196fd8fe07c2476 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:57:06.396589 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c83e1-d4db-4e09-9ac2-40ced8b71c3b service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:06.398639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c32c83e1-d4db-4e09-9ac2-40ced8b71c3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:06.403506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5624261-2a6c-47eb-8c68-2607160675d4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:06.405716 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c5624261-2a6c-47eb-8c68-2607160675d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:06.435894 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c83e1-d4db-4e09-9ac2-40ced8b71c3b service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:06.538219 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5624261-2a6c-47eb-8c68-2607160675d4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:06.543485 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c28b07-27de-4b1d-b185-45e328917182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:06.544317 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c28b07-27de-4b1d-b185-45e328917182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:06.544519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c28b07-27de-4b1d-b185-45e328917182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:06.545406 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c28b07-27de-4b1d-b185-45e328917182 admin admin] PATH_INFO: `/v3/projects/4876ec1d3559442484595f10e412ca58` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:06.545536 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847878de-bcec-44d1-97a6-67ee1ba87f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:06.545934 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847878de-bcec-44d1-97a6-67ee1ba87f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:06.546420 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847878de-bcec-44d1-97a6-67ee1ba87f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:06.546543 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c28b07-27de-4b1d-b185-45e328917182 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4876ec1d3559442484595f10e412ca58)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:06.546735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847878de-bcec-44d1-97a6-67ee1ba87f53 admin admin] PATH_INFO: `/v3/projects/734da144219a4a40bd6e058d5ca4669c` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:06.548780 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c28b07-27de-4b1d-b185-45e328917182 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:06.548985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847878de-bcec-44d1-97a6-67ee1ba87f53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=734da144219a4a40bd6e058d5ca4669c)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:06.552241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847878de-bcec-44d1-97a6-67ee1ba87f53 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:06.627051 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3c28b07-27de-4b1d-b185-45e328917182 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4876ec1d3559442484595f10e412ca58', 'request_id': 'req-e3c28b07-27de-4b1d-b185-45e328917182'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:06.642883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3c28b07-27de-4b1d-b185-45e328917182 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4876ec1d3559442484595f10e412ca58', 'request_id': 'req-e3c28b07-27de-4b1d-b185-45e328917182'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:06.653242 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:06.659326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3c28b07-27de-4b1d-b185-45e328917182 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4876ec1d3559442484595f10e412ca58', 'request_id': 'req-e3c28b07-27de-4b1d-b185-45e328917182'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:06.661850 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-847878de-bcec-44d1-97a6-67ee1ba87f53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '734da144219a4a40bd6e058d5ca4669c', 'request_id': 'req-847878de-bcec-44d1-97a6-67ee1ba87f53'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:06.671172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-847878de-bcec-44d1-97a6-67ee1ba87f53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '734da144219a4a40bd6e058d5ca4669c', 'request_id': 'req-847878de-bcec-44d1-97a6-67ee1ba87f53'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:06.685010 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-847878de-bcec-44d1-97a6-67ee1ba87f53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '734da144219a4a40bd6e058d5ca4669c', 'request_id': 'req-847878de-bcec-44d1-97a6-67ee1ba87f53'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:06.691400 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3c28b07-27de-4b1d-b185-45e328917182 admin admin] The token cache is being invalidate because project 4876ec1d3559442484595f10e412ca58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:57:06.693090 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3c28b07-27de-4b1d-b185-45e328917182 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:06.693305 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3c28b07-27de-4b1d-b185-45e328917182 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:06.695445 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1106/2215] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:57:06 2022] DELETE /identity/v3/projects/4876ec1d3559442484595f10e412ca58 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:57:06.735750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-847878de-bcec-44d1-97a6-67ee1ba87f53 admin admin] The token cache is being invalidate because project 734da144219a4a40bd6e058d5ca4669c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:57:06.736507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-847878de-bcec-44d1-97a6-67ee1ba87f53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:06.737688 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-847878de-bcec-44d1-97a6-67ee1ba87f53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:06.741329 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1110/2216] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:57:06 2022] DELETE /identity/v3/projects/734da144219a4a40bd6e058d5ca4669c => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:57:06.749956 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6920c-710e-4640-8e50-8500125bbe05 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:06.750252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6920c-710e-4640-8e50-8500125bbe05 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:06.750510 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6920c-710e-4640-8e50-8500125bbe05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:06.787920 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-5af6920c-710e-4640-8e50-8500125bbe05 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:06.789673 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5af6920c-710e-4640-8e50-8500125bbe05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:06.901533 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:06.917880 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1107/2217] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:57:06.928114 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ce9aa-8b8d-42db-8e9d-08448a61ea55 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:06.929528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ce9aa-8b8d-42db-8e9d-08448a61ea55 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:06.929528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ce9aa-8b8d-42db-8e9d-08448a61ea55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:06.962370 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cc9ce9aa-8b8d-42db-8e9d-08448a61ea55 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:06.963808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cc9ce9aa-8b8d-42db-8e9d-08448a61ea55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:06.972868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1111/2218] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:57:06.977114 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6920c-710e-4640-8e50-8500125bbe05 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:06.988124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6920c-710e-4640-8e50-8500125bbe05 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:06.994331 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f37ceb-af68-47df-a69a-8cda2fd3ed1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:06.995017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f37ceb-af68-47df-a69a-8cda2fd3ed1e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:06.995418 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f37ceb-af68-47df-a69a-8cda2fd3ed1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:06.995802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f37ceb-af68-47df-a69a-8cda2fd3ed1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:06.996654 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f37ceb-af68-47df-a69a-8cda2fd3ed1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:57:06.997083 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f37ceb-af68-47df-a69a-8cda2fd3ed1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:06.998791 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f37ceb-af68-47df-a69a-8cda2fd3ed1e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:07.015977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1108/2219] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:57:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:57:07.017002 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ce9aa-8b8d-42db-8e9d-08448a61ea55 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:07.029649 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ce9aa-8b8d-42db-8e9d-08448a61ea55 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:07.039289 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9dc3b-2c2f-41b1-9d9d-e5e703ee72b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:07.040783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9dc3b-2c2f-41b1-9d9d-e5e703ee72b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:07.041183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9dc3b-2c2f-41b1-9d9d-e5e703ee72b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:07.041794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9dc3b-2c2f-41b1-9d9d-e5e703ee72b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:07.042305 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d9dc3b-2c2f-41b1-9d9d-e5e703ee72b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:07.044469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d9dc3b-2c2f-41b1-9d9d-e5e703ee72b5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:07.077279 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1112/2220] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:57:07 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 07:57:07.081811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f37ceb-af68-47df-a69a-8cda2fd3ed1e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:07.094114 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f37ceb-af68-47df-a69a-8cda2fd3ed1e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:07.180477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de34c2-b2fe-4737-b1ab-f45b91d7ac52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:07.181231 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de34c2-b2fe-4737-b1ab-f45b91d7ac52 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:07.181542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de34c2-b2fe-4737-b1ab-f45b91d7ac52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:07.182001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de34c2-b2fe-4737-b1ab-f45b91d7ac52 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:07.182621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de34c2-b2fe-4737-b1ab-f45b91d7ac52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:07.185323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de34c2-b2fe-4737-b1ab-f45b91d7ac52 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:07.220853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95782a95-c7be-430b-9c04-8f43be472429 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:07.221208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95782a95-c7be-430b-9c04-8f43be472429 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:07.221683 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95782a95-c7be-430b-9c04-8f43be472429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:07.225787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1109/2221] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:57:07 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:57:07.231100 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de34c2-b2fe-4737-b1ab-f45b91d7ac52 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:07.241398 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-95782a95-c7be-430b-9c04-8f43be472429 None None] MFA Rules not processed for user `55b97e954da94f909f8f5ffff515509e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:07.242365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de34c2-b2fe-4737-b1ab-f45b91d7ac52 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:07.242841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-95782a95-c7be-430b-9c04-8f43be472429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:07.247219 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec5d04-87e7-436a-8ee0-331222351a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:07.247818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec5d04-87e7-436a-8ee0-331222351a6d admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:07.248110 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec5d04-87e7-436a-8ee0-331222351a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:07.248394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec5d04-87e7-436a-8ee0-331222351a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:07.249019 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec5d04-87e7-436a-8ee0-331222351a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:07.250664 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec5d04-87e7-436a-8ee0-331222351a6d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:07.265095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1110/2222] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:57:07.268353 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec5d04-87e7-436a-8ee0-331222351a6d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:07.280665 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec5d04-87e7-436a-8ee0-331222351a6d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:07.285526 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc1e23-9be0-48d5-a06f-3eb2eab9ed25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:07.286744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc1e23-9be0-48d5-a06f-3eb2eab9ed25 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:07.287166 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc1e23-9be0-48d5-a06f-3eb2eab9ed25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:07.287437 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc1e23-9be0-48d5-a06f-3eb2eab9ed25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:07.288064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc1e23-9be0-48d5-a06f-3eb2eab9ed25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:07.289698 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc1e23-9be0-48d5-a06f-3eb2eab9ed25 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:07.303772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1111/2223] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:57:07.304839 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc1e23-9be0-48d5-a06f-3eb2eab9ed25 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:07.307927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1113/2224] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:57:07.319069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc1e23-9be0-48d5-a06f-3eb2eab9ed25 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:07.324492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0960c-890b-4e2b-bca5-c7c5494d0810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:07.325148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0960c-890b-4e2b-bca5-c7c5494d0810 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:07.325421 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0960c-890b-4e2b-bca5-c7c5494d0810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:07.325692 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0960c-890b-4e2b-bca5-c7c5494d0810 admin admin] PATH_INFO: `/v3/projects/d9d921b45575486aad38303d17d65d33/users/c1c1f621292b4b95aacfea05e43df088/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:07.344150 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0960c-890b-4e2b-bca5-c7c5494d0810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9d921b45575486aad38303d17d65d33, user_id=c1c1f621292b4b95aacfea05e43df088, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:07.347040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0960c-890b-4e2b-bca5-c7c5494d0810 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:07.370953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1112/2225] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:57:07 2022] PUT /identity/v3/projects/d9d921b45575486aad38303d17d65d33/users/c1c1f621292b4b95aacfea05e43df088/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:57:07.433350 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708822c4-9687-4db4-8a6b-e29bfb9448bb None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:07.433879 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708822c4-9687-4db4-8a6b-e29bfb9448bb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:07.433879 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708822c4-9687-4db4-8a6b-e29bfb9448bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:07.464895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-708822c4-9687-4db4-8a6b-e29bfb9448bb None None] MFA Rules not processed for user `c1c1f621292b4b95aacfea05e43df088`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:07.466107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-708822c4-9687-4db4-8a6b-e29bfb9448bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:07.551400 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1114/2226] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:57:07.559666 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0960c-890b-4e2b-bca5-c7c5494d0810 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:07.562115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-61b0960c-890b-4e2b-bca5-c7c5494d0810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:07.637668 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0960c-890b-4e2b-bca5-c7c5494d0810 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:07.644573 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7af5e-7398-4dec-8308-d45bf49c0f97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:07.645831 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7af5e-7398-4dec-8308-d45bf49c0f97 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:07.646142 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7af5e-7398-4dec-8308-d45bf49c0f97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:07.646416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7af5e-7398-4dec-8308-d45bf49c0f97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:07.656455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7af5e-7398-4dec-8308-d45bf49c0f97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:07.657891 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7af5e-7398-4dec-8308-d45bf49c0f97 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:07.668985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1113/2227] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:57:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:57:17.543720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=77,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:17.662889 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=84,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:24.737517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708822c4-9687-4db4-8a6b-e29bfb9448bb None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:24.802710 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708822c4-9687-4db4-8a6b-e29bfb9448bb None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:24.807187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b4561-5271-41eb-9950-3cd949b7c59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:24.807812 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b4561-5271-41eb-9950-3cd949b7c59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:24.808112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b4561-5271-41eb-9950-3cd949b7c59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:24.808383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b4561-5271-41eb-9950-3cd949b7c59d admin admin] PATH_INFO: `/v3/users/c1c1f621292b4b95aacfea05e43df088` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:24.809502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b4561-5271-41eb-9950-3cd949b7c59d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c1f621292b4b95aacfea05e43df088)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:24.811382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b4561-5271-41eb-9950-3cd949b7c59d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:24.860756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-110b4561-5271-41eb-9950-3cd949b7c59d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c1f621292b4b95aacfea05e43df088'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:24.874708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-110b4561-5271-41eb-9950-3cd949b7c59d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c1f621292b4b95aacfea05e43df088'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:24.884739 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-110b4561-5271-41eb-9950-3cd949b7c59d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c1f621292b4b95aacfea05e43df088'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:24.885395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-110b4561-5271-41eb-9950-3cd949b7c59d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c1f621292b4b95aacfea05e43df088'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:24.894058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-110b4561-5271-41eb-9950-3cd949b7c59d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c1f621292b4b95aacfea05e43df088'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:24.901404 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1115/2228] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:57:24 2022] DELETE /identity/v3/users/c1c1f621292b4b95aacfea05e43df088 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:57:24.905713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068f067-9dbd-428c-9538-534261a07c65 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:24.906179 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068f067-9dbd-428c-9538-534261a07c65 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:24.906317 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068f067-9dbd-428c-9538-534261a07c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:24.927657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9068f067-9dbd-428c-9538-534261a07c65 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:24.929040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9068f067-9dbd-428c-9538-534261a07c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:25.002308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1114/2229] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:57:25.012852 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b4561-5271-41eb-9950-3cd949b7c59d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:25.014381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-110b4561-5271-41eb-9950-3cd949b7c59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:25.089468 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b4561-5271-41eb-9950-3cd949b7c59d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:25.095716 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8986c4-44dd-49d7-bd85-ba43dcfdcb3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:25.096547 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8986c4-44dd-49d7-bd85-ba43dcfdcb3c service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:25.096894 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8986c4-44dd-49d7-bd85-ba43dcfdcb3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:25.097282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8986c4-44dd-49d7-bd85-ba43dcfdcb3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:25.108676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8986c4-44dd-49d7-bd85-ba43dcfdcb3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:25.110194 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8986c4-44dd-49d7-bd85-ba43dcfdcb3c service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:25.121037 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1116/2230] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:57:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:57:25.463245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068f067-9dbd-428c-9538-534261a07c65 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:25.464708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9068f067-9dbd-428c-9538-534261a07c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:25.479928 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068f067-9dbd-428c-9538-534261a07c65 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:25.484592 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df31ba-3f7d-424f-b311-d8c5870da7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:25.485222 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df31ba-3f7d-424f-b311-d8c5870da7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:25.485628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df31ba-3f7d-424f-b311-d8c5870da7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:25.485906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df31ba-3f7d-424f-b311-d8c5870da7cf admin admin] PATH_INFO: `/v3/projects/d9d921b45575486aad38303d17d65d33` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:25.487589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df31ba-3f7d-424f-b311-d8c5870da7cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9d921b45575486aad38303d17d65d33)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:25.490359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df31ba-3f7d-424f-b311-d8c5870da7cf admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:25.539480 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-69df31ba-3f7d-424f-b311-d8c5870da7cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9d921b45575486aad38303d17d65d33', 'request_id': 'req-69df31ba-3f7d-424f-b311-d8c5870da7cf'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:25.544916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-69df31ba-3f7d-424f-b311-d8c5870da7cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9d921b45575486aad38303d17d65d33', 'request_id': 'req-69df31ba-3f7d-424f-b311-d8c5870da7cf'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:25.551801 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-69df31ba-3f7d-424f-b311-d8c5870da7cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9d921b45575486aad38303d17d65d33', 'request_id': 'req-69df31ba-3f7d-424f-b311-d8c5870da7cf'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:25.564839 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-69df31ba-3f7d-424f-b311-d8c5870da7cf admin admin] The token cache is being invalidate because project d9d921b45575486aad38303d17d65d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:57:25.565125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-69df31ba-3f7d-424f-b311-d8c5870da7cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:25.566001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-69df31ba-3f7d-424f-b311-d8c5870da7cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:25.567521 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1115/2231] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:57:25 2022] DELETE /identity/v3/projects/d9d921b45575486aad38303d17d65d33 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:57:25.572877 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448268a4-8509-4fdf-9d99-83c3f92cfd04 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:25.573166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448268a4-8509-4fdf-9d99-83c3f92cfd04 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:25.573526 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448268a4-8509-4fdf-9d99-83c3f92cfd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:25.595214 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-448268a4-8509-4fdf-9d99-83c3f92cfd04 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:25.596575 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-448268a4-8509-4fdf-9d99-83c3f92cfd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:25.615636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:25.668913 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1117/2232] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:57:25.677638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436952d-ab59-4eee-a7ba-30111aec68c9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:25.677958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436952d-ab59-4eee-a7ba-30111aec68c9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:25.678238 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436952d-ab59-4eee-a7ba-30111aec68c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:25.699206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-0436952d-ab59-4eee-a7ba-30111aec68c9 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:25.700475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0436952d-ab59-4eee-a7ba-30111aec68c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:25.709606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1116/2233] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:57:25.715337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448268a4-8509-4fdf-9d99-83c3f92cfd04 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:25.725951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448268a4-8509-4fdf-9d99-83c3f92cfd04 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:25.730472 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f1fdd-89b6-4bd9-8bd5-39b93e4917ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:25.731079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f1fdd-89b6-4bd9-8bd5-39b93e4917ba admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:25.731352 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f1fdd-89b6-4bd9-8bd5-39b93e4917ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:25.731614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f1fdd-89b6-4bd9-8bd5-39b93e4917ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:25.732272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f1fdd-89b6-4bd9-8bd5-39b93e4917ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 07:57:25.732569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f1fdd-89b6-4bd9-8bd5-39b93e4917ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:25.734133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f1fdd-89b6-4bd9-8bd5-39b93e4917ba admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:25.746542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1118/2234] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 07:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 07:57:25.750646 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436952d-ab59-4eee-a7ba-30111aec68c9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:25.760511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436952d-ab59-4eee-a7ba-30111aec68c9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:25.765247 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adc26f-a610-4d9f-8e25-6b2b52369fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:25.765854 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adc26f-a610-4d9f-8e25-6b2b52369fbc admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:25.766139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adc26f-a610-4d9f-8e25-6b2b52369fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:25.766482 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adc26f-a610-4d9f-8e25-6b2b52369fbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:25.767116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52adc26f-a610-4d9f-8e25-6b2b52369fbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:25.769061 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52adc26f-a610-4d9f-8e25-6b2b52369fbc admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:25.797794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1117/2235] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:57:25 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:57:25.802527 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f1fdd-89b6-4bd9-8bd5-39b93e4917ba admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:25.812937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f1fdd-89b6-4bd9-8bd5-39b93e4917ba admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:25.879151 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ba930-1d95-46cc-8dd6-cca1b52d711a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:25.879841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ba930-1d95-46cc-8dd6-cca1b52d711a admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:25.880105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ba930-1d95-46cc-8dd6-cca1b52d711a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:25.880394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ba930-1d95-46cc-8dd6-cca1b52d711a admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:25.881022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ba930-1d95-46cc-8dd6-cca1b52d711a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:25.882900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ba930-1d95-46cc-8dd6-cca1b52d711a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:25.904386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1119/2236] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:57:25 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:57:25.908328 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adc26f-a610-4d9f-8e25-6b2b52369fbc admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:25.919009 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adc26f-a610-4d9f-8e25-6b2b52369fbc admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:25.923829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0245a-4bd3-4e64-916a-786aab2fd1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:25.924690 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0245a-4bd3-4e64-916a-786aab2fd1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:25.924968 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0245a-4bd3-4e64-916a-786aab2fd1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:25.925236 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0245a-4bd3-4e64-916a-786aab2fd1c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:25.926403 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e0245a-4bd3-4e64-916a-786aab2fd1c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:25.928011 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e0245a-4bd3-4e64-916a-786aab2fd1c9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:25.938259 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1118/2237] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:57:25.943416 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ba930-1d95-46cc-8dd6-cca1b52d711a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:25.953767 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ba930-1d95-46cc-8dd6-cca1b52d711a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:25.958648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ff3e0-3f21-467d-9f15-fbf02976f546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:25.959243 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ff3e0-3f21-467d-9f15-fbf02976f546 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:25.959521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ff3e0-3f21-467d-9f15-fbf02976f546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:25.959819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ff3e0-3f21-467d-9f15-fbf02976f546 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:25.960486 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ff3e0-3f21-467d-9f15-fbf02976f546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:25.962058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ff3e0-3f21-467d-9f15-fbf02976f546 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:25.976623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1120/2238] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 07:57:25.977929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0245a-4bd3-4e64-916a-786aab2fd1c9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:25.988991 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0245a-4bd3-4e64-916a-786aab2fd1c9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:25.994652 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cb1df-5f1b-47dc-a894-7a1adf959361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:25.995570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cb1df-5f1b-47dc-a894-7a1adf959361 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:25.995979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cb1df-5f1b-47dc-a894-7a1adf959361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:25.996406 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cb1df-5f1b-47dc-a894-7a1adf959361 admin admin] PATH_INFO: `/v3/projects/53d04fe2eaaf43fc82d82f344faea612/users/8ad6564a2e9a43c2bdb0358622ff3415/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:26.013768 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952cb1df-5f1b-47dc-a894-7a1adf959361 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53d04fe2eaaf43fc82d82f344faea612, user_id=8ad6564a2e9a43c2bdb0358622ff3415, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:26.016541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952cb1df-5f1b-47dc-a894-7a1adf959361 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:26.036124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1119/2239] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:57:25 2022] PUT /identity/v3/projects/53d04fe2eaaf43fc82d82f344faea612/users/8ad6564a2e9a43c2bdb0358622ff3415/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:57:26.041268 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eeaf39-b52c-49a8-88fa-77f881cd17df None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:26.041566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eeaf39-b52c-49a8-88fa-77f881cd17df None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:26.041854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eeaf39-b52c-49a8-88fa-77f881cd17df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:26.071492 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-17eeaf39-b52c-49a8-88fa-77f881cd17df None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:26.073345 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-17eeaf39-b52c-49a8-88fa-77f881cd17df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:26.173384 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1121/2240] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:57:26.184123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cb1df-5f1b-47dc-a894-7a1adf959361 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:26.186063 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-952cb1df-5f1b-47dc-a894-7a1adf959361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:26.214304 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:26.258495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cb1df-5f1b-47dc-a894-7a1adf959361 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:26.262385 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b306d-fc16-4474-846d-232776641652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:26.262982 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b306d-fc16-4474-846d-232776641652 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:26.263272 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b306d-fc16-4474-846d-232776641652 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:26.263574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b306d-fc16-4474-846d-232776641652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:26.274190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b306d-fc16-4474-846d-232776641652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:26.275995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b306d-fc16-4474-846d-232776641652 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:26.287467 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1120/2241] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:57:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:57:28.355957 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eeaf39-b52c-49a8-88fa-77f881cd17df None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:28.357943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-17eeaf39-b52c-49a8-88fa-77f881cd17df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:28.380455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eeaf39-b52c-49a8-88fa-77f881cd17df None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:28.391900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068605cc-a446-47a1-ad3a-2873c6c5395f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:28.393471 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068605cc-a446-47a1-ad3a-2873c6c5395f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:28.393907 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068605cc-a446-47a1-ad3a-2873c6c5395f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:28.394631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068605cc-a446-47a1-ad3a-2873c6c5395f admin admin] PATH_INFO: `/v3/users/975dceb0432e428aa212effd6cff4132` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:28.397619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068605cc-a446-47a1-ad3a-2873c6c5395f admin admin] RBAC: Authorizing `identity:delete_user(user_id=975dceb0432e428aa212effd6cff4132)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:28.399818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068605cc-a446-47a1-ad3a-2873c6c5395f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:28.431250 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:28.470102 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-068605cc-a446-47a1-ad3a-2873c6c5395f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '975dceb0432e428aa212effd6cff4132'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:28.486111 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-068605cc-a446-47a1-ad3a-2873c6c5395f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '975dceb0432e428aa212effd6cff4132'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:28.496123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-068605cc-a446-47a1-ad3a-2873c6c5395f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '975dceb0432e428aa212effd6cff4132'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:28.496677 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-068605cc-a446-47a1-ad3a-2873c6c5395f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '975dceb0432e428aa212effd6cff4132'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:28.504920 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-068605cc-a446-47a1-ad3a-2873c6c5395f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '975dceb0432e428aa212effd6cff4132'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:28.512131 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1122/2242] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:57:28 2022] DELETE /identity/v3/users/975dceb0432e428aa212effd6cff4132 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:57:28.516838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b892d4-8d82-4e90-aa0c-058b1c1b5210 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:28.517363 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b892d4-8d82-4e90-aa0c-058b1c1b5210 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:28.517363 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b892d4-8d82-4e90-aa0c-058b1c1b5210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:28.541544 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-79b892d4-8d82-4e90-aa0c-058b1c1b5210 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:28.543236 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-79b892d4-8d82-4e90-aa0c-058b1c1b5210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:28.618831 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1121/2243] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:57:28.629973 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068605cc-a446-47a1-ad3a-2873c6c5395f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:28.631212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-068605cc-a446-47a1-ad3a-2873c6c5395f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:28.702216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068605cc-a446-47a1-ad3a-2873c6c5395f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:28.706216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62387ee-da5b-4209-8614-aa46b9afe37d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:28.707544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62387ee-da5b-4209-8614-aa46b9afe37d service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:28.707902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62387ee-da5b-4209-8614-aa46b9afe37d service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:28.708293 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62387ee-da5b-4209-8614-aa46b9afe37d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:28.720677 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62387ee-da5b-4209-8614-aa46b9afe37d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:28.722228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62387ee-da5b-4209-8614-aa46b9afe37d service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:28.734265 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1123/2244] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:57:29.297952 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b892d4-8d82-4e90-aa0c-058b1c1b5210 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:29.299592 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-79b892d4-8d82-4e90-aa0c-058b1c1b5210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:29.320462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b892d4-8d82-4e90-aa0c-058b1c1b5210 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:29.335233 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5e95a-d444-492e-8a3c-ee1802cc494f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:29.336315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5e95a-d444-492e-8a3c-ee1802cc494f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:29.336809 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5e95a-d444-492e-8a3c-ee1802cc494f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:29.337293 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5e95a-d444-492e-8a3c-ee1802cc494f admin admin] PATH_INFO: `/v3/projects/118f441eca754d858a8820d433d5c916` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:29.338815 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5e95a-d444-492e-8a3c-ee1802cc494f admin admin] RBAC: Authorizing `identity:delete_project(project_id=118f441eca754d858a8820d433d5c916)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:29.341087 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5e95a-d444-492e-8a3c-ee1802cc494f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:29.481085 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-12e5e95a-d444-492e-8a3c-ee1802cc494f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '118f441eca754d858a8820d433d5c916', 'request_id': 'req-12e5e95a-d444-492e-8a3c-ee1802cc494f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:29.492411 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-12e5e95a-d444-492e-8a3c-ee1802cc494f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '118f441eca754d858a8820d433d5c916', 'request_id': 'req-12e5e95a-d444-492e-8a3c-ee1802cc494f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:29.508129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-12e5e95a-d444-492e-8a3c-ee1802cc494f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '118f441eca754d858a8820d433d5c916', 'request_id': 'req-12e5e95a-d444-492e-8a3c-ee1802cc494f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:29.564471 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-12e5e95a-d444-492e-8a3c-ee1802cc494f admin admin] The token cache is being invalidate because project 118f441eca754d858a8820d433d5c916 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:57:29.565511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-12e5e95a-d444-492e-8a3c-ee1802cc494f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:29.568615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-12e5e95a-d444-492e-8a3c-ee1802cc494f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:57:29.571409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1122/2245] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:57:29 2022] DELETE /identity/v3/projects/118f441eca754d858a8820d433d5c916 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:57:32.136008 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4b60b-b900-4a54-aae7-c15612dc4768 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:32.136917 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4b60b-b900-4a54-aae7-c15612dc4768 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:32.137375 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4b60b-b900-4a54-aae7-c15612dc4768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:32.170856 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-85e4b60b-b900-4a54-aae7-c15612dc4768 None None] MFA Rules not processed for user `8ad6564a2e9a43c2bdb0358622ff3415`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:32.172673 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-85e4b60b-b900-4a54-aae7-c15612dc4768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:32.316144 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1124/2246] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:57:32.318755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5e95a-d444-492e-8a3c-ee1802cc494f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:32.324668 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-12e5e95a-d444-492e-8a3c-ee1802cc494f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:32.421673 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:32.454446 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5e95a-d444-492e-8a3c-ee1802cc494f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:32.461049 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bc903-11c8-402f-8c2a-aa3318343dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:32.461669 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bc903-11c8-402f-8c2a-aa3318343dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:32.461988 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bc903-11c8-402f-8c2a-aa3318343dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:32.462259 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bc903-11c8-402f-8c2a-aa3318343dc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:32.462902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bc903-11c8-402f-8c2a-aa3318343dc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:32.464833 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bc903-11c8-402f-8c2a-aa3318343dc4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:32.497344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1123/2247] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 07:57:32 2022] POST /identity/v3/projects => generated 371 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:57:32.501781 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4b60b-b900-4a54-aae7-c15612dc4768 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:32.518389 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4b60b-b900-4a54-aae7-c15612dc4768 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:32.557143 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:32.594072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece2801-06d3-4cea-926f-b42eb0151234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:32.594841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece2801-06d3-4cea-926f-b42eb0151234 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:32.595098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece2801-06d3-4cea-926f-b42eb0151234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:32.595449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece2801-06d3-4cea-926f-b42eb0151234 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:32.596181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece2801-06d3-4cea-926f-b42eb0151234 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:32.598575 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece2801-06d3-4cea-926f-b42eb0151234 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:32.628782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1125/2248] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 07:57:32 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 07:57:32.630180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bc903-11c8-402f-8c2a-aa3318343dc4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:32.641561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bc903-11c8-402f-8c2a-aa3318343dc4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:32.650346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b315d8e-f699-4dcb-9c96-b08e267e05e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:32.651621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b315d8e-f699-4dcb-9c96-b08e267e05e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:32.651621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b315d8e-f699-4dcb-9c96-b08e267e05e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:32.651839 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b315d8e-f699-4dcb-9c96-b08e267e05e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:32.652374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b315d8e-f699-4dcb-9c96-b08e267e05e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:32.654647 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b315d8e-f699-4dcb-9c96-b08e267e05e2 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:32.666373 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1124/2249] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:57:32.677459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece2801-06d3-4cea-926f-b42eb0151234 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:32.689610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece2801-06d3-4cea-926f-b42eb0151234 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:32.698100 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd200f3-3587-4e42-9bcb-1a387c32aaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:32.698782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd200f3-3587-4e42-9bcb-1a387c32aaae admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:32.699058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd200f3-3587-4e42-9bcb-1a387c32aaae admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:32.699329 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd200f3-3587-4e42-9bcb-1a387c32aaae admin admin] PATH_INFO: `/v3/projects/62c1491c506848a188fe63ad13c7cf47/users/db13c6e71b31467da98c6ad3da63e063/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:32.728381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd200f3-3587-4e42-9bcb-1a387c32aaae admin admin] RBAC: Authorizing `identity:create_grant(project_id=62c1491c506848a188fe63ad13c7cf47, user_id=db13c6e71b31467da98c6ad3da63e063, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:32.728381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd200f3-3587-4e42-9bcb-1a387c32aaae admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:32.742982 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1126/2250] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:57:32 2022] PUT /identity/v3/projects/62c1491c506848a188fe63ad13c7cf47/users/db13c6e71b31467da98c6ad3da63e063/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:57:32.746699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b315d8e-f699-4dcb-9c96-b08e267e05e2 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:32.758338 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b315d8e-f699-4dcb-9c96-b08e267e05e2 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:32.775522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd200f3-3587-4e42-9bcb-1a387c32aaae admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:32.777911 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5fd200f3-3587-4e42-9bcb-1a387c32aaae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:32.831991 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9ccc6-4a63-411e-92b6-bce01922bcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:32.832769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9ccc6-4a63-411e-92b6-bce01922bcb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:32.833096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9ccc6-4a63-411e-92b6-bce01922bcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:32.833304 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9ccc6-4a63-411e-92b6-bce01922bcb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:32.834011 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9ccc6-4a63-411e-92b6-bce01922bcb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:32.835630 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9ccc6-4a63-411e-92b6-bce01922bcb1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:32.845385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd200f3-3587-4e42-9bcb-1a387c32aaae admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:32.846219 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1125/2251] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 07:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 07:57:32.850654 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926ce12-e236-4e30-a660-f9b9bc95ef63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:32.850936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9ccc6-4a63-411e-92b6-bce01922bcb1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:32.851440 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926ce12-e236-4e30-a660-f9b9bc95ef63 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:32.851730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926ce12-e236-4e30-a660-f9b9bc95ef63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:32.852003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926ce12-e236-4e30-a660-f9b9bc95ef63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:32.853989 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c926ce12-e236-4e30-a660-f9b9bc95ef63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:32.862526 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9ccc6-4a63-411e-92b6-bce01922bcb1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:32.869013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c010020-09e8-4ea7-9b06-b0c6b14b8d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:32.869953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c010020-09e8-4ea7-9b06-b0c6b14b8d2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:32.870238 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c010020-09e8-4ea7-9b06-b0c6b14b8d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:32.870855 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c010020-09e8-4ea7-9b06-b0c6b14b8d2f admin admin] PATH_INFO: `/v3/projects/62c1491c506848a188fe63ad13c7cf47/users/db13c6e71b31467da98c6ad3da63e063/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:32.872866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c010020-09e8-4ea7-9b06-b0c6b14b8d2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=62c1491c506848a188fe63ad13c7cf47, user_id=db13c6e71b31467da98c6ad3da63e063, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:32.876130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c010020-09e8-4ea7-9b06-b0c6b14b8d2f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:32.889330 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1126/2252] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 07:57:32 2022] PUT /identity/v3/projects/62c1491c506848a188fe63ad13c7cf47/users/db13c6e71b31467da98c6ad3da63e063/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:57:32.937213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926ce12-e236-4e30-a660-f9b9bc95ef63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:32.938733 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926ce12-e236-4e30-a660-f9b9bc95ef63 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:32.951230 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1127/2253] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:57:32 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 07:57:39.866130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2633044-c263-4fae-beb3-164430e6c362 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:39.866681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2633044-c263-4fae-beb3-164430e6c362 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:39.866681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2633044-c263-4fae-beb3-164430e6c362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:39.903589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f2633044-c263-4fae-beb3-164430e6c362 None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:39.905146 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f2633044-c263-4fae-beb3-164430e6c362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:40.082885 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1127/2254] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Sep 7 07:57:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:57:40.123147 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926ce12-e236-4e30-a660-f9b9bc95ef63 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:40.124774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c926ce12-e236-4e30-a660-f9b9bc95ef63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:40.276627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:40.342035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926ce12-e236-4e30-a660-f9b9bc95ef63 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:40.505199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ea626-516c-4e62-9276-188904e0b614 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:40.505825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ea626-516c-4e62-9276-188904e0b614 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:40.506104 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ea626-516c-4e62-9276-188904e0b614 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:40.506375 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ea626-516c-4e62-9276-188904e0b614 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:40.523386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ea626-516c-4e62-9276-188904e0b614 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:40.525738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ea626-516c-4e62-9276-188904e0b614 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:40.538721 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1128/2255] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 07:57:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 07:57:40.625001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdb2c3-e830-4238-bee4-ef30b27fe78c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:40.625796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdb2c3-e830-4238-bee4-ef30b27fe78c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:40.626267 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdb2c3-e830-4238-bee4-ef30b27fe78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:40.683305 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9abdb2c3-e830-4238-bee4-ef30b27fe78c None None] MFA Rules not processed for user `db13c6e71b31467da98c6ad3da63e063`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:40.684548 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9abdb2c3-e830-4238-bee4-ef30b27fe78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:40.787713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:40.892512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1128/2256] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:57:40.913994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ea626-516c-4e62-9276-188904e0b614 service swift] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:40.915459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9c7ea626-516c-4e62-9276-188904e0b614 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:41.094515 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ea626-516c-4e62-9276-188904e0b614 service swift] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:41.104582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47693797-429c-47c3-86b7-b9f1db817e55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:41.105426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47693797-429c-47c3-86b7-b9f1db817e55 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:41.106542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47693797-429c-47c3-86b7-b9f1db817e55 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:41.106919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47693797-429c-47c3-86b7-b9f1db817e55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:41.198266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47693797-429c-47c3-86b7-b9f1db817e55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:41.200735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47693797-429c-47c3-86b7-b9f1db817e55 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:41.215655 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1129/2257] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:57:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:57:42.676383 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b02bc-727f-4876-aff2-6ed69954c62e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:42.677312 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b02bc-727f-4876-aff2-6ed69954c62e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:42.677594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b02bc-727f-4876-aff2-6ed69954c62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:42.705949 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b00b02bc-727f-4876-aff2-6ed69954c62e None None] MFA Rules not processed for user `e07bc40275734f628fcc2f62dd4dd0e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:42.707512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b00b02bc-727f-4876-aff2-6ed69954c62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:42.795375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1129/2258] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:57:43.044937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab4d64-a741-487b-bc11-2bc3b577c6fc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:43.045571 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab4d64-a741-487b-bc11-2bc3b577c6fc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:43.046019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab4d64-a741-487b-bc11-2bc3b577c6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:43.073013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c8ab4d64-a741-487b-bc11-2bc3b577c6fc None None] MFA Rules not processed for user `8ad6564a2e9a43c2bdb0358622ff3415`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:43.074598 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c8ab4d64-a741-487b-bc11-2bc3b577c6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:43.087874 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1130/2259] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:57:52.778190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=37 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:53.066919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:57:55.464220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a87b79-d822-433b-9e3f-6654d72497d9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:55.464713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a87b79-d822-433b-9e3f-6654d72497d9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:55.464864 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a87b79-d822-433b-9e3f-6654d72497d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:55.486003 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-93a87b79-d822-433b-9e3f-6654d72497d9 None None] MFA Rules not processed for user `b52ececfc19b41b5b8387dc6908d4a4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:55.487869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-93a87b79-d822-433b-9e3f-6654d72497d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:55.541857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1130/2260] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:57:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:57:59.653655 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e776943-ae5f-4cbf-8f3f-75a8160615ff None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:59.654191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e776943-ae5f-4cbf-8f3f-75a8160615ff None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:59.654191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e776943-ae5f-4cbf-8f3f-75a8160615ff None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:59.655560 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1131/2261] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 7 07:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:57:59.663851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66260a2f-6845-476b-a2ee-a0f079fee14b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:59.664376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66260a2f-6845-476b-a2ee-a0f079fee14b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:59.664469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66260a2f-6845-476b-a2ee-a0f079fee14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:59.688722 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-66260a2f-6845-476b-a2ee-a0f079fee14b None None] MFA Rules not processed for user `81d5689e9a344bafb74ca423ffd2b42a`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:57:59.690518 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-66260a2f-6845-476b-a2ee-a0f079fee14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:59.701240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1131/2262] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 7 07:57:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:57:59.729067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e776943-ae5f-4cbf-8f3f-75a8160615ff None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:57:59.731105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1e776943-ae5f-4cbf-8f3f-75a8160615ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:57:59.809951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e776943-ae5f-4cbf-8f3f-75a8160615ff None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:57:59.814526 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f962343-f892-44e4-87e9-538030d5ae57 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b0652c89c9945e2866512c9da4793ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:57:59.814690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f962343-f892-44e4-87e9-538030d5ae57 service glance] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:59.815133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f962343-f892-44e4-87e9-538030d5ae57 service glance] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:59.815787 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f962343-f892-44e4-87e9-538030d5ae57 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:59.855284 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f962343-f892-44e4-87e9-538030d5ae57 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:57:59.857860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f962343-f892-44e4-87e9-538030d5ae57 service glance] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:57:59.881364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1132/2263] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:57:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:57:59.987093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf29d40-f629-4498-94d3-5bcb2e1baa7f None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:59.987371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf29d40-f629-4498-94d3-5bcb2e1baa7f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:59.987851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf29d40-f629-4498-94d3-5bcb2e1baa7f None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:57:59.989224 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1132/2264] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 7 07:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:57:59.994813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ceff3-4684-433f-86b8-6d0ee18c236f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:57:59.995566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ceff3-4684-433f-86b8-6d0ee18c236f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:57:59.995848 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ceff3-4684-433f-86b8-6d0ee18c236f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:58:00.017230 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-359ceff3-4684-433f-86b8-6d0ee18c236f None None] MFA Rules not processed for user `330a38822d33439191773e9a8b4cbe04`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:58:00.018957 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-359ceff3-4684-433f-86b8-6d0ee18c236f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:58:00.087749 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1133/2265] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 7 07:57:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:58:00.102938 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf29d40-f629-4498-94d3-5bcb2e1baa7f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:58:00.120677 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf29d40-f629-4498-94d3-5bcb2e1baa7f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:58:00.124679 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf02da-c69a-461d-a70b-b11cdc0572c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:58:00.125289 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf02da-c69a-461d-a70b-b11cdc0572c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:58:00.125563 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf02da-c69a-461d-a70b-b11cdc0572c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:58:00.125832 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf02da-c69a-461d-a70b-b11cdc0572c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:58:00.135153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf02da-c69a-461d-a70b-b11cdc0572c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:58:00.136676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf02da-c69a-461d-a70b-b11cdc0572c7 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:58:00.150940 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1133/2266] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:58:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:58:00.240390 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf08a1-8fda-4670-a811-b719be6362b1 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:58:00.240679 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf08a1-8fda-4670-a811-b719be6362b1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:58:00.240975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf08a1-8fda-4670-a811-b719be6362b1 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:58:00.242361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1134/2267] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 7 07:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 07:58:00.248195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd981e5-ed00-480a-bb0c-8abf2b55ce3c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:58:00.248479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd981e5-ed00-480a-bb0c-8abf2b55ce3c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:58:00.249258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd981e5-ed00-480a-bb0c-8abf2b55ce3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:58:00.269274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-8bd981e5-ed00-480a-bb0c-8abf2b55ce3c None None] MFA Rules not processed for user `81d5689e9a344bafb74ca423ffd2b42a`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:58:00.270407 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8bd981e5-ed00-480a-bb0c-8abf2b55ce3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:58:00.279589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1134/2268] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 7 07:58:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:58:00.290748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf08a1-8fda-4670-a811-b719be6362b1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:58:00.302561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf08a1-8fda-4670-a811-b719be6362b1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:58:00.306226 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcddf77-d134-40d5-b306-e7dc97eae6b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:58:00.306832 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcddf77-d134-40d5-b306-e7dc97eae6b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:58:00.307099 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcddf77-d134-40d5-b306-e7dc97eae6b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:58:00.307425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcddf77-d134-40d5-b306-e7dc97eae6b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:58:00.318546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcddf77-d134-40d5-b306-e7dc97eae6b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:58:00.319975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcddf77-d134-40d5-b306-e7dc97eae6b9 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:58:00.333394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1135/2269] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:58:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:58:06.921005 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80817f64-3da2-4e7f-800b-7b37a6c86dd5 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:58:06.921492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80817f64-3da2-4e7f-800b-7b37a6c86dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:58:06.922438 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80817f64-3da2-4e7f-800b-7b37a6c86dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:58:06.945835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-80817f64-3da2-4e7f-800b-7b37a6c86dd5 None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:58:06.948013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-80817f64-3da2-4e7f-800b-7b37a6c86dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:58:06.957009 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1135/2270] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Sep 7 07:58:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:58:06.969637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcddf77-d134-40d5-b306-e7dc97eae6b9 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:58:06.985540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcddf77-d134-40d5-b306-e7dc97eae6b9 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:58:06.993045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389ac36-6db9-4439-898c-fcb4b027fea7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:58:06.993645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389ac36-6db9-4439-898c-fcb4b027fea7 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:58:06.993918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389ac36-6db9-4439-898c-fcb4b027fea7 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:58:06.994191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389ac36-6db9-4439-898c-fcb4b027fea7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:58:07.004506 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389ac36-6db9-4439-898c-fcb4b027fea7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:58:07.006216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389ac36-6db9-4439-898c-fcb4b027fea7 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:58:07.027253 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1136/2271] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 07:58:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 07:58:16.939910 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=40 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:58:17.011906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=61 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:58:32.672865 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277262ba-c9c7-49ab-bce9-854a4e9b3185 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:58:32.673955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277262ba-c9c7-49ab-bce9-854a4e9b3185 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:58:32.673955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277262ba-c9c7-49ab-bce9-854a4e9b3185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:58:32.693812 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-277262ba-c9c7-49ab-bce9-854a4e9b3185 None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:58:32.695006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-277262ba-c9c7-49ab-bce9-854a4e9b3185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:58:32.702111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1136/2272] 10.222.0.26 () {58 vars in 967 bytes} [Wed Sep 7 07:58:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:58:32.715324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389ac36-6db9-4439-898c-fcb4b027fea7 service swift] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:58:32.726814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389ac36-6db9-4439-898c-fcb4b027fea7 service swift] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:58:32.731835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a404359-6271-45b5-8988-bcdb20f2297c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:58:32.732452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a404359-6271-45b5-8988-bcdb20f2297c service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:58:32.732728 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a404359-6271-45b5-8988-bcdb20f2297c service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:58:32.732993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a404359-6271-45b5-8988-bcdb20f2297c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:58:32.741665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a404359-6271-45b5-8988-bcdb20f2297c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:58:32.743206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a404359-6271-45b5-8988-bcdb20f2297c service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:58:32.751273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1137/2273] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 07:58:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 07:58:37.087821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277262ba-c9c7-49ab-bce9-854a4e9b3185 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:58:37.089248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-277262ba-c9c7-49ab-bce9-854a4e9b3185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:58:37.100557 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277262ba-c9c7-49ab-bce9-854a4e9b3185 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:58:37.104015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54e26d-3923-462e-9999-ca273baf7771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:58:37.104641 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54e26d-3923-462e-9999-ca273baf7771 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:58:37.104915 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54e26d-3923-462e-9999-ca273baf7771 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:58:37.105182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54e26d-3923-462e-9999-ca273baf7771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:58:37.114595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54e26d-3923-462e-9999-ca273baf7771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:58:37.115998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54e26d-3923-462e-9999-ca273baf7771 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:58:37.125208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1137/2274] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:58:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:58:42.747957 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:58:47.121078 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=9 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:59:02.786172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a404359-6271-45b5-8988-bcdb20f2297c service swift] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:59:02.787624 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7a404359-6271-45b5-8988-bcdb20f2297c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:59:02.863678 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a404359-6271-45b5-8988-bcdb20f2297c service swift] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:59:02.869058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3dc34-5771-4218-b27f-65950bc32eca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:59:02.869667 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3dc34-5771-4218-b27f-65950bc32eca service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:59:02.869943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3dc34-5771-4218-b27f-65950bc32eca service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:59:02.870230 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3dc34-5771-4218-b27f-65950bc32eca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:59:02.871949 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ece3dc34-5771-4218-b27f-65950bc32eca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:59:02.884211 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3dc34-5771-4218-b27f-65950bc32eca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:59:02.885808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3dc34-5771-4218-b27f-65950bc32eca service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:59:02.896962 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1138/2275] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 07:59:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 07:59:12.891790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=32 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:59:14.562602 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe4078-baab-444a-989f-22fc69e5f38f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:59:14.563098 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe4078-baab-444a-989f-22fc69e5f38f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:59:14.563198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe4078-baab-444a-989f-22fc69e5f38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:59:14.582207 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d6fe4078-baab-444a-989f-22fc69e5f38f None None] MFA Rules not processed for user `e07bc40275734f628fcc2f62dd4dd0e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:59:14.583354 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d6fe4078-baab-444a-989f-22fc69e5f38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:59:14.590082 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1138/2276] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:59:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:59:24.577222 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:59:37.250557 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046d6b4-9b11-431b-afd2-9d29bf30347c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:59:37.251059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046d6b4-9b11-431b-afd2-9d29bf30347c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:59:37.251157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046d6b4-9b11-431b-afd2-9d29bf30347c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:59:37.270373 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b046d6b4-9b11-431b-afd2-9d29bf30347c None None] MFA Rules not processed for user `457d69a7306d46cd8445f887eb95c517`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:59:37.271575 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b046d6b4-9b11-431b-afd2-9d29bf30347c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:59:37.325150 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1139/2277] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:59:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:59:40.452660 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6a086-c071-4205-881b-c2c3cea21c8f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:59:40.453477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6a086-c071-4205-881b-c2c3cea21c8f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:59:40.453911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6a086-c071-4205-881b-c2c3cea21c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:59:40.477979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-69c6a086-c071-4205-881b-c2c3cea21c8f None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:59:40.479315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-69c6a086-c071-4205-881b-c2c3cea21c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:59:40.487509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1139/2278] 10.222.0.26 () {58 vars in 967 bytes} [Wed Sep 7 07:59:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 07:59:40.546335 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046d6b4-9b11-431b-afd2-9d29bf30347c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:59:40.559471 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046d6b4-9b11-431b-afd2-9d29bf30347c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:59:40.565394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce751ef-19b3-441a-9662-afbc09359165 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:59:40.566011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce751ef-19b3-441a-9662-afbc09359165 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:59:40.566296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce751ef-19b3-441a-9662-afbc09359165 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:59:40.566585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce751ef-19b3-441a-9662-afbc09359165 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:59:40.576170 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce751ef-19b3-441a-9662-afbc09359165 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:59:40.578356 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce751ef-19b3-441a-9662-afbc09359165 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:59:40.594582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1140/2279] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 07:59:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 07:59:48.573211 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6a086-c071-4205-881b-c2c3cea21c8f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:59:48.574720 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-69c6a086-c071-4205-881b-c2c3cea21c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:59:48.643114 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6a086-c071-4205-881b-c2c3cea21c8f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:59:48.647700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863d4fc-b686-4bbe-b945-cd375e6d23b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:59:48.648358 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863d4fc-b686-4bbe-b945-cd375e6d23b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:59:48.648635 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863d4fc-b686-4bbe-b945-cd375e6d23b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:59:48.648906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863d4fc-b686-4bbe-b945-cd375e6d23b3 admin admin] PATH_INFO: `/v3/users/f018f290f36f4121becdfcc2bc10827c` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:59:48.650042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863d4fc-b686-4bbe-b945-cd375e6d23b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f018f290f36f4121becdfcc2bc10827c)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:59:48.651972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863d4fc-b686-4bbe-b945-cd375e6d23b3 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:59:48.699908 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d863d4fc-b686-4bbe-b945-cd375e6d23b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f018f290f36f4121becdfcc2bc10827c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:48.700515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d863d4fc-b686-4bbe-b945-cd375e6d23b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f018f290f36f4121becdfcc2bc10827c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:48.712788 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d863d4fc-b686-4bbe-b945-cd375e6d23b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f018f290f36f4121becdfcc2bc10827c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:48.713414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d863d4fc-b686-4bbe-b945-cd375e6d23b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f018f290f36f4121becdfcc2bc10827c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:48.720189 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d863d4fc-b686-4bbe-b945-cd375e6d23b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f018f290f36f4121becdfcc2bc10827c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:48.733132 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1140/2280] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:59:48 2022] DELETE /identity/v3/users/f018f290f36f4121becdfcc2bc10827c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 07:59:48.907935 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce751ef-19b3-441a-9662-afbc09359165 service swift] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:59:48.922467 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce751ef-19b3-441a-9662-afbc09359165 service swift] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:59:48.980383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f23a3-c345-4f7e-959d-32633c227d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:59:48.981003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f23a3-c345-4f7e-959d-32633c227d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:59:48.981277 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f23a3-c345-4f7e-959d-32633c227d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:59:48.981548 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f23a3-c345-4f7e-959d-32633c227d84 admin admin] PATH_INFO: `/v3/projects/126a90736fb3438ab05ffeae4bafa7a1` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:59:48.983270 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f23a3-c345-4f7e-959d-32633c227d84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=126a90736fb3438ab05ffeae4bafa7a1)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:59:48.985194 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f23a3-c345-4f7e-959d-32633c227d84 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:59:49.030674 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-069f23a3-c345-4f7e-959d-32633c227d84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '126a90736fb3438ab05ffeae4bafa7a1', 'request_id': 'req-069f23a3-c345-4f7e-959d-32633c227d84'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:49.036238 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-069f23a3-c345-4f7e-959d-32633c227d84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '126a90736fb3438ab05ffeae4bafa7a1', 'request_id': 'req-069f23a3-c345-4f7e-959d-32633c227d84'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:49.042955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-069f23a3-c345-4f7e-959d-32633c227d84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '126a90736fb3438ab05ffeae4bafa7a1', 'request_id': 'req-069f23a3-c345-4f7e-959d-32633c227d84'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:49.055571 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-069f23a3-c345-4f7e-959d-32633c227d84 admin admin] The token cache is being invalidate because project 126a90736fb3438ab05ffeae4bafa7a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:59:49.055853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-069f23a3-c345-4f7e-959d-32633c227d84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:49.056484 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-069f23a3-c345-4f7e-959d-32633c227d84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:49.057939 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1141/2281] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:59:48 2022] DELETE /identity/v3/projects/126a90736fb3438ab05ffeae4bafa7a1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:59:49.062685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863d4fc-b686-4bbe-b945-cd375e6d23b3 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:59:49.064413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d863d4fc-b686-4bbe-b945-cd375e6d23b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:59:49.077781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863d4fc-b686-4bbe-b945-cd375e6d23b3 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:59:49.138071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c81042-1920-4e15-be38-0747f0e5d384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:59:49.139056 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c81042-1920-4e15-be38-0747f0e5d384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:59:49.139571 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c81042-1920-4e15-be38-0747f0e5d384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:59:49.139884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c81042-1920-4e15-be38-0747f0e5d384 admin admin] PATH_INFO: `/v3/users/935523b362544defa4596ebff6c8aa82` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:59:49.141288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c81042-1920-4e15-be38-0747f0e5d384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=935523b362544defa4596ebff6c8aa82)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:59:49.145555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c81042-1920-4e15-be38-0747f0e5d384 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:59:49.166004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:59:49.207682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a8c81042-1920-4e15-be38-0747f0e5d384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '935523b362544defa4596ebff6c8aa82'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:49.208472 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a8c81042-1920-4e15-be38-0747f0e5d384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '935523b362544defa4596ebff6c8aa82'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:49.221077 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a8c81042-1920-4e15-be38-0747f0e5d384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '935523b362544defa4596ebff6c8aa82'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:49.221737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a8c81042-1920-4e15-be38-0747f0e5d384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '935523b362544defa4596ebff6c8aa82'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:49.228687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a8c81042-1920-4e15-be38-0747f0e5d384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '935523b362544defa4596ebff6c8aa82'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:49.243782 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1141/2282] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 07:59:49 2022] DELETE /identity/v3/users/935523b362544defa4596ebff6c8aa82 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:59:49.439634 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f23a3-c345-4f7e-959d-32633c227d84 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 07:59:49.443217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-069f23a3-c345-4f7e-959d-32633c227d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:59:49.457584 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f23a3-c345-4f7e-959d-32633c227d84 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 07:59:49.484035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 07:59:49.543383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505677d-c51b-4645-bcbd-f3c2aa918ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 07:59:49.544377 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505677d-c51b-4645-bcbd-f3c2aa918ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:59:49.545434 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505677d-c51b-4645-bcbd-f3c2aa918ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:59:49.545434 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505677d-c51b-4645-bcbd-f3c2aa918ca5 admin admin] PATH_INFO: `/v3/projects/6574ad7065204f13a74970802ca48371` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:59:49.546872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505677d-c51b-4645-bcbd-f3c2aa918ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6574ad7065204f13a74970802ca48371)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 07:59:49.548867 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505677d-c51b-4645-bcbd-f3c2aa918ca5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 07:59:49.599112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e505677d-c51b-4645-bcbd-f3c2aa918ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6574ad7065204f13a74970802ca48371', 'request_id': 'req-e505677d-c51b-4645-bcbd-f3c2aa918ca5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:49.604934 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e505677d-c51b-4645-bcbd-f3c2aa918ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6574ad7065204f13a74970802ca48371', 'request_id': 'req-e505677d-c51b-4645-bcbd-f3c2aa918ca5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:49.611794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e505677d-c51b-4645-bcbd-f3c2aa918ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6574ad7065204f13a74970802ca48371', 'request_id': 'req-e505677d-c51b-4645-bcbd-f3c2aa918ca5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:49.628643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e505677d-c51b-4645-bcbd-f3c2aa918ca5 admin admin] The token cache is being invalidate because project 6574ad7065204f13a74970802ca48371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 07:59:49.628936 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e505677d-c51b-4645-bcbd-f3c2aa918ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:49.629580 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e505677d-c51b-4645-bcbd-f3c2aa918ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 07:59:49.631075 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1142/2283] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 07:59:49 2022] DELETE /identity/v3/projects/6574ad7065204f13a74970802ca48371 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 07:59:52.882122 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba4184-c4ae-4c19-a5e3-dcc00f784ebe None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 07:59:52.882647 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba4184-c4ae-4c19-a5e3-dcc00f784ebe None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 07:59:52.882758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba4184-c4ae-4c19-a5e3-dcc00f784ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 07:59:52.902830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-4fba4184-c4ae-4c19-a5e3-dcc00f784ebe None None] MFA Rules not processed for user `b52ececfc19b41b5b8387dc6908d4a4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 07:59:52.904025 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4fba4184-c4ae-4c19-a5e3-dcc00f784ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 07:59:52.956916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1142/2284] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 07:59:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 07:59:59.625297 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:00:02.950152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=46,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:00:13.867289 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505677d-c51b-4645-bcbd-f3c2aa918ca5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:00:13.868703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e505677d-c51b-4645-bcbd-f3c2aa918ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:13.926424 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505677d-c51b-4645-bcbd-f3c2aa918ca5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:00:13.930067 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5f695-c2e2-4399-a3ca-fb01f3e53b2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:00:13.930667 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5f695-c2e2-4399-a3ca-fb01f3e53b2c service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:00:13.930945 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5f695-c2e2-4399-a3ca-fb01f3e53b2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:00:13.931246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5f695-c2e2-4399-a3ca-fb01f3e53b2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:00:13.933138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-acc5f695-c2e2-4399-a3ca-fb01f3e53b2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:14.003453 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5f695-c2e2-4399-a3ca-fb01f3e53b2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:00:14.004933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5f695-c2e2-4399-a3ca-fb01f3e53b2c service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:00:14.016442 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1143/2285] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:00:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:00:21.853494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba4184-c4ae-4c19-a5e3-dcc00f784ebe None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:00:21.863254 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba4184-c4ae-4c19-a5e3-dcc00f784ebe None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:00:21.866839 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a608ff3-6be0-49a1-87e1-7768eaaced5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:00:21.867459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a608ff3-6be0-49a1-87e1-7768eaaced5e service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:00:21.867722 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a608ff3-6be0-49a1-87e1-7768eaaced5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:00:21.867991 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a608ff3-6be0-49a1-87e1-7768eaaced5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:00:21.869716 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2a608ff3-6be0-49a1-87e1-7768eaaced5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:21.883143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a608ff3-6be0-49a1-87e1-7768eaaced5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:00:21.884614 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a608ff3-6be0-49a1-87e1-7768eaaced5e service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:00:21.896063 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1143/2286] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:00:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:00:24.010963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=54 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:00:28.333502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5f695-c2e2-4399-a3ca-fb01f3e53b2c service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:00:28.335003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-acc5f695-c2e2-4399-a3ca-fb01f3e53b2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:28.349557 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5f695-c2e2-4399-a3ca-fb01f3e53b2c service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:00:28.355023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1bbbe-f226-47c5-a5ec-a62bd05062b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:00:28.355623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1bbbe-f226-47c5-a5ec-a62bd05062b8 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:00:28.355901 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1bbbe-f226-47c5-a5ec-a62bd05062b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:00:28.356215 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1bbbe-f226-47c5-a5ec-a62bd05062b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:00:28.358171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-69d1bbbe-f226-47c5-a5ec-a62bd05062b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:28.417156 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1bbbe-f226-47c5-a5ec-a62bd05062b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:00:28.418740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1bbbe-f226-47c5-a5ec-a62bd05062b8 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:00:28.428204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1144/2287] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:00:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:00:30.321132 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0842e-1668-4fa0-8554-be1bbd0d462e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:00:30.321628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0842e-1668-4fa0-8554-be1bbd0d462e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:00:30.321628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0842e-1668-4fa0-8554-be1bbd0d462e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:00:30.343521 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-dda0842e-1668-4fa0-8554-be1bbd0d462e None None] MFA Rules not processed for user `e07bc40275734f628fcc2f62dd4dd0e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:00:30.344858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dda0842e-1668-4fa0-8554-be1bbd0d462e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:30.351689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1144/2288] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:00:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:00:30.656935 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1bbbe-f226-47c5-a5ec-a62bd05062b8 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:00:30.658347 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-69d1bbbe-f226-47c5-a5ec-a62bd05062b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:30.713948 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1bbbe-f226-47c5-a5ec-a62bd05062b8 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:00:30.718260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd267ff7-4546-46c5-b21c-56dd3e6ddcdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:00:30.718854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd267ff7-4546-46c5-b21c-56dd3e6ddcdf service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:00:30.719132 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd267ff7-4546-46c5-b21c-56dd3e6ddcdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:00:30.719407 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd267ff7-4546-46c5-b21c-56dd3e6ddcdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:00:30.721137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fd267ff7-4546-46c5-b21c-56dd3e6ddcdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:30.730295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd267ff7-4546-46c5-b21c-56dd3e6ddcdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:00:30.731702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd267ff7-4546-46c5-b21c-56dd3e6ddcdf service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:00:30.740040 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1145/2289] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:00:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:00:31.055601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0842e-1668-4fa0-8554-be1bbd0d462e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:00:31.057025 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dda0842e-1668-4fa0-8554-be1bbd0d462e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:31.127898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0842e-1668-4fa0-8554-be1bbd0d462e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:00:31.133082 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01767aaa-1d76-411e-a59b-1c1f2db26c11 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:00:31.133680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01767aaa-1d76-411e-a59b-1c1f2db26c11 service placement] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:00:31.133957 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01767aaa-1d76-411e-a59b-1c1f2db26c11 service placement] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:00:31.134226 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01767aaa-1d76-411e-a59b-1c1f2db26c11 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:00:31.135899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-01767aaa-1d76-411e-a59b-1c1f2db26c11 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:31.147814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01767aaa-1d76-411e-a59b-1c1f2db26c11 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:00:31.149418 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01767aaa-1d76-411e-a59b-1c1f2db26c11 service placement] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:00:31.159131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1145/2290] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:00:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:00:34.771551 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd267ff7-4546-46c5-b21c-56dd3e6ddcdf service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:00:34.773019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fd267ff7-4546-46c5-b21c-56dd3e6ddcdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:34.789211 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd267ff7-4546-46c5-b21c-56dd3e6ddcdf service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:00:34.794532 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a20221-a2a0-4f4c-b8fa-c4e2a6c230a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:00:34.795115 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a20221-a2a0-4f4c-b8fa-c4e2a6c230a5 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:00:34.795395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a20221-a2a0-4f4c-b8fa-c4e2a6c230a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:00:34.795659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a20221-a2a0-4f4c-b8fa-c4e2a6c230a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:00:34.797325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-81a20221-a2a0-4f4c-b8fa-c4e2a6c230a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:34.854144 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a20221-a2a0-4f4c-b8fa-c4e2a6c230a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:00:34.855690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a20221-a2a0-4f4c-b8fa-c4e2a6c230a5 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:00:34.864673 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1146/2291] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:00:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:00:35.908258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01767aaa-1d76-411e-a59b-1c1f2db26c11 service placement] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:00:35.918423 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01767aaa-1d76-411e-a59b-1c1f2db26c11 service placement] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:00:35.922719 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a7ecb-5ad5-439a-bb46-be4bd4b7dab2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:00:35.923314 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a7ecb-5ad5-439a-bb46-be4bd4b7dab2 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:00:35.923586 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a7ecb-5ad5-439a-bb46-be4bd4b7dab2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:00:35.923853 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a7ecb-5ad5-439a-bb46-be4bd4b7dab2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:00:35.925656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1b2a7ecb-5ad5-439a-bb46-be4bd4b7dab2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:35.936718 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a7ecb-5ad5-439a-bb46-be4bd4b7dab2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:00:35.938735 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a7ecb-5ad5-439a-bb46-be4bd4b7dab2 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:00:35.947747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1146/2292] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:00:35 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:00:44.409029 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a20221-a2a0-4f4c-b8fa-c4e2a6c230a5 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:00:44.421380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a20221-a2a0-4f4c-b8fa-c4e2a6c230a5 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:00:44.425814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e6a2a-c540-4db3-a08a-1414edab8b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:00:44.426517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e6a2a-c540-4db3-a08a-1414edab8b52 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:00:44.426860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e6a2a-c540-4db3-a08a-1414edab8b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:00:44.427209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e6a2a-c540-4db3-a08a-1414edab8b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:00:44.429428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7f4e6a2a-c540-4db3-a08a-1414edab8b52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:44.441153 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e6a2a-c540-4db3-a08a-1414edab8b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:00:44.442562 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e6a2a-c540-4db3-a08a-1414edab8b52 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:00:44.450756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1147/2293] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:00:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:00:45.945640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=47 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:00:46.163062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a7ecb-5ad5-439a-bb46-be4bd4b7dab2 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:00:46.179219 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a7ecb-5ad5-439a-bb46-be4bd4b7dab2 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:00:46.185219 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c03981-dcd1-4b82-814f-df3c23322742 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:00:46.186872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c03981-dcd1-4b82-814f-df3c23322742 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:00:46.187393 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c03981-dcd1-4b82-814f-df3c23322742 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:00:46.187887 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c03981-dcd1-4b82-814f-df3c23322742 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:00:46.191322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e6c03981-dcd1-4b82-814f-df3c23322742 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:46.208614 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c03981-dcd1-4b82-814f-df3c23322742 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:00:46.210464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c03981-dcd1-4b82-814f-df3c23322742 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:00:46.239359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1147/2294] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:00:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:00:49.058583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e6a2a-c540-4db3-a08a-1414edab8b52 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:00:49.077800 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e6a2a-c540-4db3-a08a-1414edab8b52 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:00:49.097723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea4df9-1db2-424e-bc5d-e3d14326a7b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:00:49.098477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea4df9-1db2-424e-bc5d-e3d14326a7b1 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:00:49.098878 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea4df9-1db2-424e-bc5d-e3d14326a7b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:00:49.099301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea4df9-1db2-424e-bc5d-e3d14326a7b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:00:49.101592 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a3ea4df9-1db2-424e-bc5d-e3d14326a7b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:49.121867 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea4df9-1db2-424e-bc5d-e3d14326a7b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:00:49.126030 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea4df9-1db2-424e-bc5d-e3d14326a7b1 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:00:49.159422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1148/2295] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:00:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:00:51.196038 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81415d6-7967-4eb9-a16c-79be54da9890 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:00:51.196576 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81415d6-7967-4eb9-a16c-79be54da9890 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:00:51.196684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81415d6-7967-4eb9-a16c-79be54da9890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:00:51.216000 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e81415d6-7967-4eb9-a16c-79be54da9890 None None] MFA Rules not processed for user `55b97e954da94f909f8f5ffff515509e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:00:51.217874 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e81415d6-7967-4eb9-a16c-79be54da9890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:51.273098 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1148/2296] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:00:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:00:51.604902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea4df9-1db2-424e-bc5d-e3d14326a7b1 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:00:51.606449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a3ea4df9-1db2-424e-bc5d-e3d14326a7b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:51.617920 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea4df9-1db2-424e-bc5d-e3d14326a7b1 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:00:51.621428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a3d93-a837-40a7-817a-db9a3050b4fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:00:51.622019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a3d93-a837-40a7-817a-db9a3050b4fd service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:00:51.622295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a3d93-a837-40a7-817a-db9a3050b4fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:00:51.622565 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a3d93-a837-40a7-817a-db9a3050b4fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:00:51.625082 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f52a3d93-a837-40a7-817a-db9a3050b4fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:00:51.634563 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a3d93-a837-40a7-817a-db9a3050b4fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:00:51.635967 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a3d93-a837-40a7-817a-db9a3050b4fd service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:00:51.640911 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:00:51.646090 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1149/2297] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:00:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:01:00.293918 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5ac0a-33a3-4c61-a596-5439bc2554e1 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:00.294365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5ac0a-33a3-4c61-a596-5439bc2554e1 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:00.294481 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5ac0a-33a3-4c61-a596-5439bc2554e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:00.316712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-84f5ac0a-33a3-4c61-a596-5439bc2554e1 None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:00.317858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-84f5ac0a-33a3-4c61-a596-5439bc2554e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:00.377455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1149/2298] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Sep 7 08:01:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:01:00.428299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a3d93-a837-40a7-817a-db9a3050b4fd service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:00.429825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f52a3d93-a837-40a7-817a-db9a3050b4fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:00.502619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a3d93-a837-40a7-817a-db9a3050b4fd service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:00.511233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65da9a-263d-42be-a086-edc126a3189d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:00.511233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65da9a-263d-42be-a086-edc126a3189d service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:00.511233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65da9a-263d-42be-a086-edc126a3189d service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:00.511233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65da9a-263d-42be-a086-edc126a3189d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:00.517817 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65da9a-263d-42be-a086-edc126a3189d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:00.519415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65da9a-263d-42be-a086-edc126a3189d service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:00.527669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1150/2299] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 08:01:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 08:01:06.188866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5ac0a-33a3-4c61-a596-5439bc2554e1 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:06.199994 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5ac0a-33a3-4c61-a596-5439bc2554e1 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:06.207655 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc5747-dbaa-4425-8128-125e387f3498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:06.208966 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc5747-dbaa-4425-8128-125e387f3498 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:06.209313 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc5747-dbaa-4425-8128-125e387f3498 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:06.209943 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc5747-dbaa-4425-8128-125e387f3498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:06.211965 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7bbc5747-dbaa-4425-8128-125e387f3498 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:06.297298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc5747-dbaa-4425-8128-125e387f3498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:06.299464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc5747-dbaa-4425-8128-125e387f3498 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:06.310953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1150/2300] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:01:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:01:07.464905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65da9a-263d-42be-a086-edc126a3189d service swift] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:07.489385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65da9a-263d-42be-a086-edc126a3189d service swift] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:07.501149 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c07afb-dffb-4f78-a6c3-7a7aabd7e0c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:07.502915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c07afb-dffb-4f78-a6c3-7a7aabd7e0c5 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:07.503532 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c07afb-dffb-4f78-a6c3-7a7aabd7e0c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:07.511462 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c07afb-dffb-4f78-a6c3-7a7aabd7e0c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:07.515274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-18c07afb-dffb-4f78-a6c3-7a7aabd7e0c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:07.529437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c07afb-dffb-4f78-a6c3-7a7aabd7e0c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:07.532109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c07afb-dffb-4f78-a6c3-7a7aabd7e0c5 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:07.550041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1151/2301] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:01:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:01:11.748022 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc5747-dbaa-4425-8128-125e387f3498 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:11.760801 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc5747-dbaa-4425-8128-125e387f3498 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:11.768737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac218571-1156-4b96-8e6b-aa9def0f36da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:11.773348 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac218571-1156-4b96-8e6b-aa9def0f36da service placement] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:11.773671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac218571-1156-4b96-8e6b-aa9def0f36da service placement] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:11.773955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac218571-1156-4b96-8e6b-aa9def0f36da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:11.775927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ac218571-1156-4b96-8e6b-aa9def0f36da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:11.788580 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac218571-1156-4b96-8e6b-aa9def0f36da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:11.790216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac218571-1156-4b96-8e6b-aa9def0f36da service placement] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:11.802533 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1151/2302] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:01:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:01:11.875868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c07afb-dffb-4f78-a6c3-7a7aabd7e0c5 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:11.877300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-18c07afb-dffb-4f78-a6c3-7a7aabd7e0c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:11.894458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c07afb-dffb-4f78-a6c3-7a7aabd7e0c5 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:11.902105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c855d-7068-41c3-b80d-1f70bb0ab7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:11.902796 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c855d-7068-41c3-b80d-1f70bb0ab7ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:11.903177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c855d-7068-41c3-b80d-1f70bb0ab7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:11.903548 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c855d-7068-41c3-b80d-1f70bb0ab7ef admin admin] PATH_INFO: `/v3/users/b52ececfc19b41b5b8387dc6908d4a4a` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:11.905636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c855d-7068-41c3-b80d-1f70bb0ab7ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=b52ececfc19b41b5b8387dc6908d4a4a)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:11.907711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c855d-7068-41c3-b80d-1f70bb0ab7ef admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:11.969153 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f78c855d-7068-41c3-b80d-1f70bb0ab7ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b52ececfc19b41b5b8387dc6908d4a4a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:11.984503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f78c855d-7068-41c3-b80d-1f70bb0ab7ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b52ececfc19b41b5b8387dc6908d4a4a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:11.992879 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f78c855d-7068-41c3-b80d-1f70bb0ab7ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b52ececfc19b41b5b8387dc6908d4a4a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:11.994371 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f78c855d-7068-41c3-b80d-1f70bb0ab7ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b52ececfc19b41b5b8387dc6908d4a4a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:12.003830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f78c855d-7068-41c3-b80d-1f70bb0ab7ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b52ececfc19b41b5b8387dc6908d4a4a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:12.011865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1152/2303] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:01:11 2022] DELETE /identity/v3/users/b52ececfc19b41b5b8387dc6908d4a4a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:01:12.232224 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac218571-1156-4b96-8e6b-aa9def0f36da service placement] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:12.244378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac218571-1156-4b96-8e6b-aa9def0f36da service placement] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:12.259057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:01:12.312663 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d96cc-5aed-4cf0-9a9d-98bbde23a07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:12.313454 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d96cc-5aed-4cf0-9a9d-98bbde23a07f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:12.313732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d96cc-5aed-4cf0-9a9d-98bbde23a07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:12.314008 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d96cc-5aed-4cf0-9a9d-98bbde23a07f admin admin] PATH_INFO: `/v3/projects/4d3d7139782f4b63ac6dc8d4618abe64` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:12.315196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d96cc-5aed-4cf0-9a9d-98bbde23a07f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d3d7139782f4b63ac6dc8d4618abe64)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:12.317171 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d96cc-5aed-4cf0-9a9d-98bbde23a07f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:12.368788 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5d9d96cc-5aed-4cf0-9a9d-98bbde23a07f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d3d7139782f4b63ac6dc8d4618abe64', 'request_id': 'req-5d9d96cc-5aed-4cf0-9a9d-98bbde23a07f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:12.375680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5d9d96cc-5aed-4cf0-9a9d-98bbde23a07f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d3d7139782f4b63ac6dc8d4618abe64', 'request_id': 'req-5d9d96cc-5aed-4cf0-9a9d-98bbde23a07f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:12.384063 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5d9d96cc-5aed-4cf0-9a9d-98bbde23a07f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d3d7139782f4b63ac6dc8d4618abe64', 'request_id': 'req-5d9d96cc-5aed-4cf0-9a9d-98bbde23a07f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:12.401198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5d9d96cc-5aed-4cf0-9a9d-98bbde23a07f admin admin] The token cache is being invalidate because project 4d3d7139782f4b63ac6dc8d4618abe64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:01:12.402027 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5d9d96cc-5aed-4cf0-9a9d-98bbde23a07f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:12.402820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5d9d96cc-5aed-4cf0-9a9d-98bbde23a07f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:12.404668 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1152/2304] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:01:12 2022] DELETE /identity/v3/projects/4d3d7139782f4b63ac6dc8d4618abe64 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:01:12.409605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21708c12-ceae-477b-88ab-ba77bdbb097b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:12.409896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21708c12-ceae-477b-88ab-ba77bdbb097b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:12.410182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21708c12-ceae-477b-88ab-ba77bdbb097b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:12.437936 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-21708c12-ceae-477b-88ab-ba77bdbb097b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:12.439443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-21708c12-ceae-477b-88ab-ba77bdbb097b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:12.487850 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:01:12.505600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1153/2305] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:01:12.513318 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d7352-b150-404e-a663-2c37830007cc None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:12.513630 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d7352-b150-404e-a663-2c37830007cc None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:12.513887 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d7352-b150-404e-a663-2c37830007cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:12.534225 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7c3d7352-b150-404e-a663-2c37830007cc None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:12.535696 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7c3d7352-b150-404e-a663-2c37830007cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:12.544612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1153/2306] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:01:12.549003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21708c12-ceae-477b-88ab-ba77bdbb097b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:12.560062 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21708c12-ceae-477b-88ab-ba77bdbb097b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:12.568291 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7485b46-d40c-410c-8797-53d5ae307b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:12.568929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7485b46-d40c-410c-8797-53d5ae307b87 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:12.569307 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7485b46-d40c-410c-8797-53d5ae307b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:12.569879 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7485b46-d40c-410c-8797-53d5ae307b87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:12.570678 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7485b46-d40c-410c-8797-53d5ae307b87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:01:12.571084 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7485b46-d40c-410c-8797-53d5ae307b87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:12.572890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7485b46-d40c-410c-8797-53d5ae307b87 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:12.591828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1154/2307] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:01:12.593568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d7352-b150-404e-a663-2c37830007cc None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:12.603962 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d7352-b150-404e-a663-2c37830007cc None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:12.610732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a98bd6-7cdd-4997-940c-acb46daacbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:12.611351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a98bd6-7cdd-4997-940c-acb46daacbf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:12.611817 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a98bd6-7cdd-4997-940c-acb46daacbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:12.612221 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a98bd6-7cdd-4997-940c-acb46daacbf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:12.612864 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a98bd6-7cdd-4997-940c-acb46daacbf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:12.614786 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a98bd6-7cdd-4997-940c-acb46daacbf4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:12.646728 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1154/2308] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:01:12 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:01:12.651946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7485b46-d40c-410c-8797-53d5ae307b87 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:12.663346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7485b46-d40c-410c-8797-53d5ae307b87 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:12.744199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ae120-d507-4a83-9bcc-7cad51d4cb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:12.745513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ae120-d507-4a83-9bcc-7cad51d4cb70 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:12.745967 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ae120-d507-4a83-9bcc-7cad51d4cb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:12.746496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ae120-d507-4a83-9bcc-7cad51d4cb70 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:12.747249 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ae120-d507-4a83-9bcc-7cad51d4cb70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:12.751130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ae120-d507-4a83-9bcc-7cad51d4cb70 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:12.811439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1155/2309] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:01:12 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:01:12.817028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a98bd6-7cdd-4997-940c-acb46daacbf4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:12.831206 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a98bd6-7cdd-4997-940c-acb46daacbf4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:12.838245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17daa7db-c29a-4c6e-834e-d8debef405d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:12.838854 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daa7db-c29a-4c6e-834e-d8debef405d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:12.839806 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daa7db-c29a-4c6e-834e-d8debef405d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:12.840096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daa7db-c29a-4c6e-834e-d8debef405d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:12.841065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17daa7db-c29a-4c6e-834e-d8debef405d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:12.843734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17daa7db-c29a-4c6e-834e-d8debef405d9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:12.862465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1155/2310] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:01:12.866267 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ae120-d507-4a83-9bcc-7cad51d4cb70 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:12.877037 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ae120-d507-4a83-9bcc-7cad51d4cb70 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:12.881990 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a836db-75d6-4116-94ff-0828d08ed193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:12.882627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a836db-75d6-4116-94ff-0828d08ed193 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:12.882913 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a836db-75d6-4116-94ff-0828d08ed193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:12.883262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a836db-75d6-4116-94ff-0828d08ed193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:12.883906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a836db-75d6-4116-94ff-0828d08ed193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:12.886277 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a836db-75d6-4116-94ff-0828d08ed193 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:12.899341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1156/2311] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:01:12.905400 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17daa7db-c29a-4c6e-834e-d8debef405d9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:12.919773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17daa7db-c29a-4c6e-834e-d8debef405d9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:12.926082 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a52763-ef59-486a-8b5a-a51afb7d2416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:12.926856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a52763-ef59-486a-8b5a-a51afb7d2416 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:12.927255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a52763-ef59-486a-8b5a-a51afb7d2416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:12.927620 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a52763-ef59-486a-8b5a-a51afb7d2416 admin admin] PATH_INFO: `/v3/projects/d30a51f45f6a4147a505a7f0a3414e5c/users/c78abc908f1f41c28940f5fe1ce656c2/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:12.944559 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a52763-ef59-486a-8b5a-a51afb7d2416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d30a51f45f6a4147a505a7f0a3414e5c, user_id=c78abc908f1f41c28940f5fe1ce656c2, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:12.947372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a52763-ef59-486a-8b5a-a51afb7d2416 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:12.972698 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1156/2312] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:01:12 2022] PUT /identity/v3/projects/d30a51f45f6a4147a505a7f0a3414e5c/users/c78abc908f1f41c28940f5fe1ce656c2/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:01:12.977422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021afbee-f99f-4219-b829-61021c41b7ff None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:12.977705 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021afbee-f99f-4219-b829-61021c41b7ff None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:12.977985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021afbee-f99f-4219-b829-61021c41b7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:13.008239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-021afbee-f99f-4219-b829-61021c41b7ff None None] MFA Rules not processed for user `c78abc908f1f41c28940f5fe1ce656c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:13.009488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-021afbee-f99f-4219-b829-61021c41b7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:13.103090 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1157/2313] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:01:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:01:13.111285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a52763-ef59-486a-8b5a-a51afb7d2416 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:13.114014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e3a52763-ef59-486a-8b5a-a51afb7d2416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:13.186767 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:01:13.227506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a52763-ef59-486a-8b5a-a51afb7d2416 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:13.235113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be1e8c-dfc7-4c21-a33e-d3e831cd99e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:13.235941 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be1e8c-dfc7-4c21-a33e-d3e831cd99e1 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:13.236350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be1e8c-dfc7-4c21-a33e-d3e831cd99e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:13.236724 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be1e8c-dfc7-4c21-a33e-d3e831cd99e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:13.246784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be1e8c-dfc7-4c21-a33e-d3e831cd99e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:13.250128 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be1e8c-dfc7-4c21-a33e-d3e831cd99e1 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:13.261460 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1157/2314] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:01:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:01:20.402295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7018bb-6bfa-477e-9d65-2311e2a309ec None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:20.402824 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7018bb-6bfa-477e-9d65-2311e2a309ec None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:20.402824 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7018bb-6bfa-477e-9d65-2311e2a309ec None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:20.404178 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1158/2315] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 7 08:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 08:01:20.410602 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c552e-b8d4-4390-a67d-11e1b696ba13 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:20.411055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c552e-b8d4-4390-a67d-11e1b696ba13 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:20.411173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c552e-b8d4-4390-a67d-11e1b696ba13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:20.431584 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d59c552e-b8d4-4390-a67d-11e1b696ba13 None None] MFA Rules not processed for user `81d5689e9a344bafb74ca423ffd2b42a`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:20.432824 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d59c552e-b8d4-4390-a67d-11e1b696ba13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:20.442814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1158/2316] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 7 08:01:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:01:20.454495 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7018bb-6bfa-477e-9d65-2311e2a309ec None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:20.456932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4c7018bb-6bfa-477e-9d65-2311e2a309ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:20.505054 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:01:20.518354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7018bb-6bfa-477e-9d65-2311e2a309ec None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:20.521941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95e761-03c8-439b-ba73-6bb4899fde75 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b0652c89c9945e2866512c9da4793ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:20.522572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95e761-03c8-439b-ba73-6bb4899fde75 service glance] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:20.522849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95e761-03c8-439b-ba73-6bb4899fde75 service glance] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:20.523122 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95e761-03c8-439b-ba73-6bb4899fde75 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:20.544033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95e761-03c8-439b-ba73-6bb4899fde75 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:20.544033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95e761-03c8-439b-ba73-6bb4899fde75 service glance] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:20.549038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1159/2317] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:01:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:01:20.629115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13af41-1176-4768-bcb9-6d83ebac200b None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:20.629462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13af41-1176-4768-bcb9-6d83ebac200b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:20.629799 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13af41-1176-4768-bcb9-6d83ebac200b None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:20.631258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1159/2318] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 7 08:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 08:01:20.636254 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed13a3-8e30-4321-9170-0e8c099f3c19 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:20.636556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed13a3-8e30-4321-9170-0e8c099f3c19 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:20.636855 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed13a3-8e30-4321-9170-0e8c099f3c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:20.659981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-36ed13a3-8e30-4321-9170-0e8c099f3c19 None None] MFA Rules not processed for user `330a38822d33439191773e9a8b4cbe04`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:20.661197 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-36ed13a3-8e30-4321-9170-0e8c099f3c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:20.673088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13af41-1176-4768-bcb9-6d83ebac200b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:20.721137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1160/2319] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 7 08:01:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:01:20.733664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed13a3-8e30-4321-9170-0e8c099f3c19 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:20.735181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-36ed13a3-8e30-4321-9170-0e8c099f3c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:20.737210 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13af41-1176-4768-bcb9-6d83ebac200b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:20.741868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684458c2-dfbb-42b6-8ffc-097075fd6e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:20.742501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684458c2-dfbb-42b6-8ffc-097075fd6e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:20.742777 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684458c2-dfbb-42b6-8ffc-097075fd6e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:20.743052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684458c2-dfbb-42b6-8ffc-097075fd6e7d admin admin] PATH_INFO: `/v3/users/c78abc908f1f41c28940f5fe1ce656c2` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:20.744227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684458c2-dfbb-42b6-8ffc-097075fd6e7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c78abc908f1f41c28940f5fe1ce656c2)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:20.745521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed13a3-8e30-4321-9170-0e8c099f3c19 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:20.746148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684458c2-dfbb-42b6-8ffc-097075fd6e7d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:20.749264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41012d7d-1349-4860-b8aa-d3769bbf9613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:20.749857 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41012d7d-1349-4860-b8aa-d3769bbf9613 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:20.750134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41012d7d-1349-4860-b8aa-d3769bbf9613 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:20.750408 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41012d7d-1349-4860-b8aa-d3769bbf9613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:20.757971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41012d7d-1349-4860-b8aa-d3769bbf9613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:20.759469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41012d7d-1349-4860-b8aa-d3769bbf9613 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:20.768301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1161/2320] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:01:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:01:20.796160 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-684458c2-dfbb-42b6-8ffc-097075fd6e7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c78abc908f1f41c28940f5fe1ce656c2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:20.796830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-684458c2-dfbb-42b6-8ffc-097075fd6e7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c78abc908f1f41c28940f5fe1ce656c2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:20.809229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-684458c2-dfbb-42b6-8ffc-097075fd6e7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c78abc908f1f41c28940f5fe1ce656c2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:20.810124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-684458c2-dfbb-42b6-8ffc-097075fd6e7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c78abc908f1f41c28940f5fe1ce656c2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:20.817740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-684458c2-dfbb-42b6-8ffc-097075fd6e7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c78abc908f1f41c28940f5fe1ce656c2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:20.832763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1160/2321] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:01:20 2022] DELETE /identity/v3/users/c78abc908f1f41c28940f5fe1ce656c2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:01:20.837291 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab29ac0-3a74-4537-a11f-5a002533b6d3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:20.837583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab29ac0-3a74-4537-a11f-5a002533b6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:20.837856 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab29ac0-3a74-4537-a11f-5a002533b6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:20.857775 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2ab29ac0-3a74-4537-a11f-5a002533b6d3 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:20.858978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2ab29ac0-3a74-4537-a11f-5a002533b6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:20.860479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cbae7b-cfe0-43e1-b537-5434706980de None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:20.860771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cbae7b-cfe0-43e1-b537-5434706980de None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:20.861079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cbae7b-cfe0-43e1-b537-5434706980de None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:20.862650 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1161/2322] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 7 08:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 08:01:20.868315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17d969-2f16-464b-9f61-2827f991bd9e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:20.868615 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17d969-2f16-464b-9f61-2827f991bd9e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:20.869045 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17d969-2f16-464b-9f61-2827f991bd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:20.892859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1b17d969-2f16-464b-9f61-2827f991bd9e None None] MFA Rules not processed for user `81d5689e9a344bafb74ca423ffd2b42a`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:20.894102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1b17d969-2f16-464b-9f61-2827f991bd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:20.919634 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1162/2323] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:01:20.930174 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab29ac0-3a74-4537-a11f-5a002533b6d3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:20.931591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2ab29ac0-3a74-4537-a11f-5a002533b6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:20.933814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:01:20.968060 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1162/2324] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 7 08:01:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:01:20.978699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17d969-2f16-464b-9f61-2827f991bd9e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:20.980110 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1b17d969-2f16-464b-9f61-2827f991bd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:21.003156 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab29ac0-3a74-4537-a11f-5a002533b6d3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:21.007162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee116af4-267f-4bfd-a9d4-db78e2ff339e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:21.008056 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee116af4-267f-4bfd-a9d4-db78e2ff339e service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:21.008525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee116af4-267f-4bfd-a9d4-db78e2ff339e service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:21.008797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee116af4-267f-4bfd-a9d4-db78e2ff339e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:21.018962 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee116af4-267f-4bfd-a9d4-db78e2ff339e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:21.020460 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee116af4-267f-4bfd-a9d4-db78e2ff339e service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:21.031088 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1163/2325] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:01:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:01:21.047909 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17d969-2f16-464b-9f61-2827f991bd9e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:21.052128 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708db3f-d83b-4749-957f-41be5a4040d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:21.052842 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708db3f-d83b-4749-957f-41be5a4040d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:21.053191 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708db3f-d83b-4749-957f-41be5a4040d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:21.053532 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708db3f-d83b-4749-957f-41be5a4040d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:21.065663 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708db3f-d83b-4749-957f-41be5a4040d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:21.067397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708db3f-d83b-4749-957f-41be5a4040d6 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:21.080501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1163/2326] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:01:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:01:21.368344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee116af4-267f-4bfd-a9d4-db78e2ff339e service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:21.369774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ee116af4-267f-4bfd-a9d4-db78e2ff339e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:21.395362 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee116af4-267f-4bfd-a9d4-db78e2ff339e service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:21.402249 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e27a6-ee92-4b7b-bffd-04b8ee83576c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:21.403205 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e27a6-ee92-4b7b-bffd-04b8ee83576c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:21.403889 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e27a6-ee92-4b7b-bffd-04b8ee83576c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:21.404065 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e27a6-ee92-4b7b-bffd-04b8ee83576c admin admin] PATH_INFO: `/v3/projects/d30a51f45f6a4147a505a7f0a3414e5c` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:21.405794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e27a6-ee92-4b7b-bffd-04b8ee83576c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d30a51f45f6a4147a505a7f0a3414e5c)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:21.407715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e27a6-ee92-4b7b-bffd-04b8ee83576c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:21.433860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:01:21.460125 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b23e27a6-ee92-4b7b-bffd-04b8ee83576c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd30a51f45f6a4147a505a7f0a3414e5c', 'request_id': 'req-b23e27a6-ee92-4b7b-bffd-04b8ee83576c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:21.465872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b23e27a6-ee92-4b7b-bffd-04b8ee83576c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd30a51f45f6a4147a505a7f0a3414e5c', 'request_id': 'req-b23e27a6-ee92-4b7b-bffd-04b8ee83576c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:21.472677 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b23e27a6-ee92-4b7b-bffd-04b8ee83576c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd30a51f45f6a4147a505a7f0a3414e5c', 'request_id': 'req-b23e27a6-ee92-4b7b-bffd-04b8ee83576c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:21.486425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b23e27a6-ee92-4b7b-bffd-04b8ee83576c admin admin] The token cache is being invalidate because project d30a51f45f6a4147a505a7f0a3414e5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:01:21.486715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b23e27a6-ee92-4b7b-bffd-04b8ee83576c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:21.487351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b23e27a6-ee92-4b7b-bffd-04b8ee83576c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:21.489055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1164/2327] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:01:21 2022] DELETE /identity/v3/projects/d30a51f45f6a4147a505a7f0a3414e5c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:01:21.493957 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf38099-6b07-4a20-82ff-23e5c22520f2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:21.494459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf38099-6b07-4a20-82ff-23e5c22520f2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:21.494561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf38099-6b07-4a20-82ff-23e5c22520f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:21.515289 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-0bf38099-6b07-4a20-82ff-23e5c22520f2 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:21.516539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0bf38099-6b07-4a20-82ff-23e5c22520f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:21.581007 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1164/2328] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:01:21.589165 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c547f3e-7a64-4baf-a43a-b01078a6c6ff None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:21.589466 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c547f3e-7a64-4baf-a43a-b01078a6c6ff None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:21.589740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c547f3e-7a64-4baf-a43a-b01078a6c6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:21.612355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7c547f3e-7a64-4baf-a43a-b01078a6c6ff None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:21.613584 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7c547f3e-7a64-4baf-a43a-b01078a6c6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:21.621756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1165/2329] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:01:21.626377 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf38099-6b07-4a20-82ff-23e5c22520f2 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:21.637450 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf38099-6b07-4a20-82ff-23e5c22520f2 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:21.644034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10826f30-cef1-4c48-9483-daa9d1c72f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:21.644602 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10826f30-cef1-4c48-9483-daa9d1c72f8a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:21.644877 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10826f30-cef1-4c48-9483-daa9d1c72f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:21.645146 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10826f30-cef1-4c48-9483-daa9d1c72f8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:21.645787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10826f30-cef1-4c48-9483-daa9d1c72f8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:01:21.646084 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10826f30-cef1-4c48-9483-daa9d1c72f8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:21.647686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10826f30-cef1-4c48-9483-daa9d1c72f8a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:21.663675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1165/2330] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:01:21.668064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c547f3e-7a64-4baf-a43a-b01078a6c6ff None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:21.677961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c547f3e-7a64-4baf-a43a-b01078a6c6ff None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:21.683553 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d0611-be51-4c6e-b22f-5a5bdef6a397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:21.684173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d0611-be51-4c6e-b22f-5a5bdef6a397 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:21.684724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d0611-be51-4c6e-b22f-5a5bdef6a397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:21.684997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d0611-be51-4c6e-b22f-5a5bdef6a397 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:21.685780 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d0611-be51-4c6e-b22f-5a5bdef6a397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:21.688481 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d0611-be51-4c6e-b22f-5a5bdef6a397 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:21.716773 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1166/2331] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:01:21 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:01:21.721033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10826f30-cef1-4c48-9483-daa9d1c72f8a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:21.731639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10826f30-cef1-4c48-9483-daa9d1c72f8a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:21.810412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f0c41-52ef-4efa-86d1-49cbd18d2363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:21.811982 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f0c41-52ef-4efa-86d1-49cbd18d2363 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:21.812334 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f0c41-52ef-4efa-86d1-49cbd18d2363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:21.812610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f0c41-52ef-4efa-86d1-49cbd18d2363 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:21.813749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f0c41-52ef-4efa-86d1-49cbd18d2363 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:21.816564 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f0c41-52ef-4efa-86d1-49cbd18d2363 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:21.828501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:01:21.843316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1166/2332] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:01:21 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:01:21.847777 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d0611-be51-4c6e-b22f-5a5bdef6a397 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:21.860117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d0611-be51-4c6e-b22f-5a5bdef6a397 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:21.866112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b58c6-86d0-4496-b991-e229e7363d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:21.866824 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b58c6-86d0-4496-b991-e229e7363d8b admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:21.867196 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b58c6-86d0-4496-b991-e229e7363d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:21.867538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b58c6-86d0-4496-b991-e229e7363d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:21.868552 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b58c6-86d0-4496-b991-e229e7363d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:21.870564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b58c6-86d0-4496-b991-e229e7363d8b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:21.884252 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1167/2333] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:01:21.890240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f0c41-52ef-4efa-86d1-49cbd18d2363 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:21.901015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f0c41-52ef-4efa-86d1-49cbd18d2363 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:21.906234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13562b9-6f56-461f-991d-f5c39c9b3de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:21.906964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13562b9-6f56-461f-991d-f5c39c9b3de6 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:21.907251 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13562b9-6f56-461f-991d-f5c39c9b3de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:21.907519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13562b9-6f56-461f-991d-f5c39c9b3de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:21.908203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13562b9-6f56-461f-991d-f5c39c9b3de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:21.909814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13562b9-6f56-461f-991d-f5c39c9b3de6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:21.923277 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1167/2334] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:01:21.927244 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b58c6-86d0-4496-b991-e229e7363d8b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:21.940449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b58c6-86d0-4496-b991-e229e7363d8b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:21.945055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661fdbe-95dd-400e-9299-eb897eca6074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:21.945545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661fdbe-95dd-400e-9299-eb897eca6074 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:21.945823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661fdbe-95dd-400e-9299-eb897eca6074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:21.946217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661fdbe-95dd-400e-9299-eb897eca6074 admin admin] PATH_INFO: `/v3/projects/d9f3057781a24109ae8eeb6e2a6e1d4b/users/49bbbe581e6745d6afc980f0d6cb0fb6/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:21.962848 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c661fdbe-95dd-400e-9299-eb897eca6074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f3057781a24109ae8eeb6e2a6e1d4b, user_id=49bbbe581e6745d6afc980f0d6cb0fb6, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:21.965559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c661fdbe-95dd-400e-9299-eb897eca6074 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:21.981147 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1168/2335] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:01:21 2022] PUT /identity/v3/projects/d9f3057781a24109ae8eeb6e2a6e1d4b/users/49bbbe581e6745d6afc980f0d6cb0fb6/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:01:21.992714 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e13792-ad77-4bfa-afd1-3f7dd2897cbf None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:21.993019 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e13792-ad77-4bfa-afd1-3f7dd2897cbf None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:21.993333 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e13792-ad77-4bfa-afd1-3f7dd2897cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:22.022353 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-08e13792-ad77-4bfa-afd1-3f7dd2897cbf None None] MFA Rules not processed for user `49bbbe581e6745d6afc980f0d6cb0fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:22.023625 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-08e13792-ad77-4bfa-afd1-3f7dd2897cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:22.087991 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1168/2336] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:01:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:01:22.092728 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661fdbe-95dd-400e-9299-eb897eca6074 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:22.153564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661fdbe-95dd-400e-9299-eb897eca6074 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:22.158138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d744b5e-5e3b-4a0e-9983-2fbf3f284fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:22.158732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d744b5e-5e3b-4a0e-9983-2fbf3f284fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:22.159012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d744b5e-5e3b-4a0e-9983-2fbf3f284fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:22.159287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d744b5e-5e3b-4a0e-9983-2fbf3f284fc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:22.159921 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d744b5e-5e3b-4a0e-9983-2fbf3f284fc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:22.161833 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d744b5e-5e3b-4a0e-9983-2fbf3f284fc0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:22.189454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1169/2337] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:01:22 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:01:22.193749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e13792-ad77-4bfa-afd1-3f7dd2897cbf None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:22.204440 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e13792-ad77-4bfa-afd1-3f7dd2897cbf None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:22.261953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d68ea-5cc2-4a77-8bb0-8c09f880d19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:22.262561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d68ea-5cc2-4a77-8bb0-8c09f880d19f admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:22.262836 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d68ea-5cc2-4a77-8bb0-8c09f880d19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:22.263106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d68ea-5cc2-4a77-8bb0-8c09f880d19f admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:22.263755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d68ea-5cc2-4a77-8bb0-8c09f880d19f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:22.265707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d68ea-5cc2-4a77-8bb0-8c09f880d19f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:22.287869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1169/2338] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:01:22 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:01:22.293149 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d744b5e-5e3b-4a0e-9983-2fbf3f284fc0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:22.307333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d744b5e-5e3b-4a0e-9983-2fbf3f284fc0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:22.312757 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a7669-de8d-4cc7-8b4e-a14794263ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:22.313771 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a7669-de8d-4cc7-8b4e-a14794263ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:22.314167 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a7669-de8d-4cc7-8b4e-a14794263ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:22.314444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a7669-de8d-4cc7-8b4e-a14794263ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:22.315080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a7669-de8d-4cc7-8b4e-a14794263ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:22.316717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a7669-de8d-4cc7-8b4e-a14794263ee6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:22.327879 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1170/2339] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:01:22.332161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d68ea-5cc2-4a77-8bb0-8c09f880d19f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:22.342721 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d68ea-5cc2-4a77-8bb0-8c09f880d19f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:22.347771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a6a78-cc2a-47b8-a792-67bd1b14d176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:22.348458 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a6a78-cc2a-47b8-a792-67bd1b14d176 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:22.348735 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a6a78-cc2a-47b8-a792-67bd1b14d176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:22.349006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a6a78-cc2a-47b8-a792-67bd1b14d176 admin admin] PATH_INFO: `/v3/projects/916743be023b4090925d2f9f0fd1e30d/users/6f195cf27f3e48719789006aab32de1c/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:22.365100 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a6a78-cc2a-47b8-a792-67bd1b14d176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=916743be023b4090925d2f9f0fd1e30d, user_id=6f195cf27f3e48719789006aab32de1c, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:22.367780 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a6a78-cc2a-47b8-a792-67bd1b14d176 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:22.382765 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1170/2340] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:01:22 2022] PUT /identity/v3/projects/916743be023b4090925d2f9f0fd1e30d/users/6f195cf27f3e48719789006aab32de1c/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:01:22.386792 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a7669-de8d-4cc7-8b4e-a14794263ee6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:22.397471 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a7669-de8d-4cc7-8b4e-a14794263ee6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:22.422969 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:01:22.467683 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0e92d-33bd-4913-9359-8504e08e549b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:22.468319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0e92d-33bd-4913-9359-8504e08e549b admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:22.468600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0e92d-33bd-4913-9359-8504e08e549b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:22.468870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0e92d-33bd-4913-9359-8504e08e549b admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:22.469503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b0e92d-33bd-4913-9359-8504e08e549b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:22.471085 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b0e92d-33bd-4913-9359-8504e08e549b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:22.481003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1171/2341] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:01:22.485836 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a6a78-cc2a-47b8-a792-67bd1b14d176 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:22.496312 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a6a78-cc2a-47b8-a792-67bd1b14d176 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:22.501812 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232e944-7146-476a-a6db-d1373386dd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:22.502470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232e944-7146-476a-a6db-d1373386dd9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:22.502742 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232e944-7146-476a-a6db-d1373386dd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:22.503009 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232e944-7146-476a-a6db-d1373386dd9c admin admin] PATH_INFO: `/v3/projects/916743be023b4090925d2f9f0fd1e30d/users/6f195cf27f3e48719789006aab32de1c/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:22.504975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232e944-7146-476a-a6db-d1373386dd9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=916743be023b4090925d2f9f0fd1e30d, user_id=6f195cf27f3e48719789006aab32de1c, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:22.507713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232e944-7146-476a-a6db-d1373386dd9c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:22.519103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1171/2342] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:01:22 2022] PUT /identity/v3/projects/916743be023b4090925d2f9f0fd1e30d/users/6f195cf27f3e48719789006aab32de1c/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:01:22.531018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14afd6e-a2c4-4d65-89b6-5346989ed5e2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:22.531302 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14afd6e-a2c4-4d65-89b6-5346989ed5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:22.531574 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14afd6e-a2c4-4d65-89b6-5346989ed5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:22.560572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b14afd6e-a2c4-4d65-89b6-5346989ed5e2 None None] MFA Rules not processed for user `6f195cf27f3e48719789006aab32de1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:22.562059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b14afd6e-a2c4-4d65-89b6-5346989ed5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:22.634021 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1172/2343] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:01:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:01:22.644956 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232e944-7146-476a-a6db-d1373386dd9c admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:22.646854 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4232e944-7146-476a-a6db-d1373386dd9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:22.676757 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:01:22.708217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232e944-7146-476a-a6db-d1373386dd9c admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:22.711867 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b8d5c-4ba3-4242-892d-7e81ab20fe5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:22.712503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b8d5c-4ba3-4242-892d-7e81ab20fe5f service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:22.712779 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b8d5c-4ba3-4242-892d-7e81ab20fe5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:22.713076 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b8d5c-4ba3-4242-892d-7e81ab20fe5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:22.771443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b8d5c-4ba3-4242-892d-7e81ab20fe5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:22.772927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b8d5c-4ba3-4242-892d-7e81ab20fe5f service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:22.783346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1172/2344] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:01:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:01:23.226581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417501e-a4a3-4f4a-a3d6-aadab885dcb3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:23.226872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417501e-a4a3-4f4a-a3d6-aadab885dcb3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:23.227152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417501e-a4a3-4f4a-a3d6-aadab885dcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:23.247840 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d417501e-a4a3-4f4a-a3d6-aadab885dcb3 None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:23.249861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d417501e-a4a3-4f4a-a3d6-aadab885dcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:23.314809 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1173/2345] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Sep 7 08:01:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:01:23.329658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b8d5c-4ba3-4242-892d-7e81ab20fe5f service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:23.331091 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-899b8d5c-4ba3-4242-892d-7e81ab20fe5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:23.409569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b8d5c-4ba3-4242-892d-7e81ab20fe5f service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:23.419395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b622e2-5d7b-4439-b53d-28fb65de0470 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:23.420146 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b622e2-5d7b-4439-b53d-28fb65de0470 service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:23.420434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b622e2-5d7b-4439-b53d-28fb65de0470 service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:23.420708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b622e2-5d7b-4439-b53d-28fb65de0470 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:23.429579 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b622e2-5d7b-4439-b53d-28fb65de0470 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:23.431170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b622e2-5d7b-4439-b53d-28fb65de0470 service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:23.441167 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1173/2346] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 08:01:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 08:01:28.514101 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417501e-a4a3-4f4a-a3d6-aadab885dcb3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:28.515573 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d417501e-a4a3-4f4a-a3d6-aadab885dcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:28.528804 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417501e-a4a3-4f4a-a3d6-aadab885dcb3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:28.532932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22036b6-ad92-4c4d-a917-0a56eeb8e54a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:28.533534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22036b6-ad92-4c4d-a917-0a56eeb8e54a service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:28.533807 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22036b6-ad92-4c4d-a917-0a56eeb8e54a service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:28.534072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22036b6-ad92-4c4d-a917-0a56eeb8e54a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:28.543716 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22036b6-ad92-4c4d-a917-0a56eeb8e54a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:28.545164 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22036b6-ad92-4c4d-a917-0a56eeb8e54a service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:28.555474 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1174/2347] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:01:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:01:33.437510 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=69 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:01:38.550781 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=85 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:01:44.729955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a43172-06b8-462e-9b66-b11a9a129419 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:44.730399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a43172-06b8-462e-9b66-b11a9a129419 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:44.730512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a43172-06b8-462e-9b66-b11a9a129419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:44.751185 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-97a43172-06b8-462e-9b66-b11a9a129419 None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:44.752443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-97a43172-06b8-462e-9b66-b11a9a129419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:44.760994 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1174/2348] 10.222.0.26 () {58 vars in 967 bytes} [Wed Sep 7 08:01:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:01:44.773446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22036b6-ad92-4c4d-a917-0a56eeb8e54a service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:44.784657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22036b6-ad92-4c4d-a917-0a56eeb8e54a service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:44.789583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512214c1-8bc6-44c5-bc88-5bf999f59c23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:44.790163 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512214c1-8bc6-44c5-bc88-5bf999f59c23 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:44.790437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512214c1-8bc6-44c5-bc88-5bf999f59c23 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:44.790706 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512214c1-8bc6-44c5-bc88-5bf999f59c23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:44.799262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512214c1-8bc6-44c5-bc88-5bf999f59c23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:44.800864 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512214c1-8bc6-44c5-bc88-5bf999f59c23 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:44.808934 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1175/2349] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 08:01:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 08:01:53.625192 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a43172-06b8-462e-9b66-b11a9a129419 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:53.692170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a43172-06b8-462e-9b66-b11a9a129419 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:53.696561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add02328-8b75-4f37-94c6-f16fb9c25586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:53.697191 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add02328-8b75-4f37-94c6-f16fb9c25586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:53.697466 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add02328-8b75-4f37-94c6-f16fb9c25586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:53.697734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add02328-8b75-4f37-94c6-f16fb9c25586 admin admin] PATH_INFO: `/v3/users/49bbbe581e6745d6afc980f0d6cb0fb6` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:53.698851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add02328-8b75-4f37-94c6-f16fb9c25586 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49bbbe581e6745d6afc980f0d6cb0fb6)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:53.700809 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add02328-8b75-4f37-94c6-f16fb9c25586 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:53.753838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-add02328-8b75-4f37-94c6-f16fb9c25586 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49bbbe581e6745d6afc980f0d6cb0fb6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:53.754844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-add02328-8b75-4f37-94c6-f16fb9c25586 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49bbbe581e6745d6afc980f0d6cb0fb6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:53.769790 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-add02328-8b75-4f37-94c6-f16fb9c25586 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49bbbe581e6745d6afc980f0d6cb0fb6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:53.770598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-add02328-8b75-4f37-94c6-f16fb9c25586 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49bbbe581e6745d6afc980f0d6cb0fb6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:53.781735 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-add02328-8b75-4f37-94c6-f16fb9c25586 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49bbbe581e6745d6afc980f0d6cb0fb6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:53.801612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1175/2350] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:01:53 2022] DELETE /identity/v3/users/49bbbe581e6745d6afc980f0d6cb0fb6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:01:53.803709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844ca83-8c5c-4aac-9edc-da79aab13d76 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:53.804235 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844ca83-8c5c-4aac-9edc-da79aab13d76 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:53.804546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844ca83-8c5c-4aac-9edc-da79aab13d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:53.828345 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6844ca83-8c5c-4aac-9edc-da79aab13d76 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:53.829608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6844ca83-8c5c-4aac-9edc-da79aab13d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:53.894083 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1176/2351] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:01:53.903680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add02328-8b75-4f37-94c6-f16fb9c25586 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:53.905913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-add02328-8b75-4f37-94c6-f16fb9c25586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:53.970092 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add02328-8b75-4f37-94c6-f16fb9c25586 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:53.973800 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da084f0-3493-4fdf-91b7-30fafc25a137 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:53.974405 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da084f0-3493-4fdf-91b7-30fafc25a137 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:53.974685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da084f0-3493-4fdf-91b7-30fafc25a137 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:53.974957 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da084f0-3493-4fdf-91b7-30fafc25a137 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:53.985940 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da084f0-3493-4fdf-91b7-30fafc25a137 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:53.987659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da084f0-3493-4fdf-91b7-30fafc25a137 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:53.999928 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1176/2352] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:01:54.542151 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844ca83-8c5c-4aac-9edc-da79aab13d76 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:54.543726 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6844ca83-8c5c-4aac-9edc-da79aab13d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:54.556101 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844ca83-8c5c-4aac-9edc-da79aab13d76 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:54.560876 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b65a8e-4071-4e82-838b-88b860a9249e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:54.562260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b65a8e-4071-4e82-838b-88b860a9249e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:54.562860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b65a8e-4071-4e82-838b-88b860a9249e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:54.563134 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b65a8e-4071-4e82-838b-88b860a9249e admin admin] PATH_INFO: `/v3/projects/d9f3057781a24109ae8eeb6e2a6e1d4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:54.564281 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b65a8e-4071-4e82-838b-88b860a9249e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f3057781a24109ae8eeb6e2a6e1d4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:54.566170 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b65a8e-4071-4e82-838b-88b860a9249e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:54.614041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f7b65a8e-4071-4e82-838b-88b860a9249e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f3057781a24109ae8eeb6e2a6e1d4b', 'request_id': 'req-f7b65a8e-4071-4e82-838b-88b860a9249e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:54.619488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f7b65a8e-4071-4e82-838b-88b860a9249e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f3057781a24109ae8eeb6e2a6e1d4b', 'request_id': 'req-f7b65a8e-4071-4e82-838b-88b860a9249e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:54.626315 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f7b65a8e-4071-4e82-838b-88b860a9249e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f3057781a24109ae8eeb6e2a6e1d4b', 'request_id': 'req-f7b65a8e-4071-4e82-838b-88b860a9249e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:54.638338 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f7b65a8e-4071-4e82-838b-88b860a9249e admin admin] The token cache is being invalidate because project d9f3057781a24109ae8eeb6e2a6e1d4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:01:54.638622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f7b65a8e-4071-4e82-838b-88b860a9249e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:54.639247 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f7b65a8e-4071-4e82-838b-88b860a9249e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:54.640872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1177/2353] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:01:54 2022] DELETE /identity/v3/projects/d9f3057781a24109ae8eeb6e2a6e1d4b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:01:54.644938 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da084f0-3493-4fdf-91b7-30fafc25a137 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:54.654339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da084f0-3493-4fdf-91b7-30fafc25a137 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:54.665548 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:01:54.711969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc9629-99c8-4526-966c-f5d7c5d66f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:54.712619 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc9629-99c8-4526-966c-f5d7c5d66f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:54.712898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc9629-99c8-4526-966c-f5d7c5d66f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:54.713171 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc9629-99c8-4526-966c-f5d7c5d66f61 admin admin] PATH_INFO: `/v3/users/6f195cf27f3e48719789006aab32de1c` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:54.714280 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc9629-99c8-4526-966c-f5d7c5d66f61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f195cf27f3e48719789006aab32de1c)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:54.716258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc9629-99c8-4526-966c-f5d7c5d66f61 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:54.763274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7edc9629-99c8-4526-966c-f5d7c5d66f61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f195cf27f3e48719789006aab32de1c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:54.763830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7edc9629-99c8-4526-966c-f5d7c5d66f61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f195cf27f3e48719789006aab32de1c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:54.775740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7edc9629-99c8-4526-966c-f5d7c5d66f61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f195cf27f3e48719789006aab32de1c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:54.776396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7edc9629-99c8-4526-966c-f5d7c5d66f61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f195cf27f3e48719789006aab32de1c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:54.783817 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7edc9629-99c8-4526-966c-f5d7c5d66f61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f195cf27f3e48719789006aab32de1c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:54.797134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1177/2354] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:01:54 2022] DELETE /identity/v3/users/6f195cf27f3e48719789006aab32de1c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:01:55.095858 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b65a8e-4071-4e82-838b-88b860a9249e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:55.097990 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f7b65a8e-4071-4e82-838b-88b860a9249e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:55.115154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b65a8e-4071-4e82-838b-88b860a9249e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:55.180199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce3218-f069-41ec-9568-32b409cf16c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:55.180830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce3218-f069-41ec-9568-32b409cf16c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:55.181107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce3218-f069-41ec-9568-32b409cf16c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:55.181368 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce3218-f069-41ec-9568-32b409cf16c7 admin admin] PATH_INFO: `/v3/projects/916743be023b4090925d2f9f0fd1e30d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:55.182476 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce3218-f069-41ec-9568-32b409cf16c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=916743be023b4090925d2f9f0fd1e30d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:55.184401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce3218-f069-41ec-9568-32b409cf16c7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:55.195951 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:01:55.235857 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-14ce3218-f069-41ec-9568-32b409cf16c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '916743be023b4090925d2f9f0fd1e30d', 'request_id': 'req-14ce3218-f069-41ec-9568-32b409cf16c7'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:55.241247 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-14ce3218-f069-41ec-9568-32b409cf16c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '916743be023b4090925d2f9f0fd1e30d', 'request_id': 'req-14ce3218-f069-41ec-9568-32b409cf16c7'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:55.249986 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-14ce3218-f069-41ec-9568-32b409cf16c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '916743be023b4090925d2f9f0fd1e30d', 'request_id': 'req-14ce3218-f069-41ec-9568-32b409cf16c7'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:55.263770 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-14ce3218-f069-41ec-9568-32b409cf16c7 admin admin] The token cache is being invalidate because project 916743be023b4090925d2f9f0fd1e30d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:01:55.264175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-14ce3218-f069-41ec-9568-32b409cf16c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:55.265447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-14ce3218-f069-41ec-9568-32b409cf16c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:01:55.267130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1178/2355] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:01:55 2022] DELETE /identity/v3/projects/916743be023b4090925d2f9f0fd1e30d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:01:55.271955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff4ad3-2472-447c-a533-7ac9d3990564 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:55.272342 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff4ad3-2472-447c-a533-7ac9d3990564 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:55.272725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff4ad3-2472-447c-a533-7ac9d3990564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:55.295539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-21ff4ad3-2472-447c-a533-7ac9d3990564 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:55.296864 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-21ff4ad3-2472-447c-a533-7ac9d3990564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:55.357935 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1178/2356] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:01:55.366111 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35257f-bd7d-40fa-b270-b39eae1d2780 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:55.366390 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35257f-bd7d-40fa-b270-b39eae1d2780 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:55.366662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35257f-bd7d-40fa-b270-b39eae1d2780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:55.395068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-aa35257f-bd7d-40fa-b270-b39eae1d2780 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:55.396356 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-aa35257f-bd7d-40fa-b270-b39eae1d2780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:55.405538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1179/2357] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:01:55.409488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff4ad3-2472-447c-a533-7ac9d3990564 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:55.419562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff4ad3-2472-447c-a533-7ac9d3990564 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:55.424526 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb256adc-bdb4-487c-a4bf-deb4e927f6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:55.425538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb256adc-bdb4-487c-a4bf-deb4e927f6ee admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:55.425826 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb256adc-bdb4-487c-a4bf-deb4e927f6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:55.426108 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb256adc-bdb4-487c-a4bf-deb4e927f6ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:55.426735 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb256adc-bdb4-487c-a4bf-deb4e927f6ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:01:55.427035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb256adc-bdb4-487c-a4bf-deb4e927f6ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:55.428665 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb256adc-bdb4-487c-a4bf-deb4e927f6ee admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:55.445896 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1179/2358] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:01:55.450315 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35257f-bd7d-40fa-b270-b39eae1d2780 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:55.461525 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35257f-bd7d-40fa-b270-b39eae1d2780 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:55.466232 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f4dc9-ca4f-473c-9a2b-447df5358027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:55.466817 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f4dc9-ca4f-473c-9a2b-447df5358027 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:55.467091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f4dc9-ca4f-473c-9a2b-447df5358027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:55.467385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f4dc9-ca4f-473c-9a2b-447df5358027 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:55.468627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f4dc9-ca4f-473c-9a2b-447df5358027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:55.471479 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f4dc9-ca4f-473c-9a2b-447df5358027 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:55.503583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1180/2359] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:01:55 2022] POST /identity/v3/projects => generated 401 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:01:55.507823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb256adc-bdb4-487c-a4bf-deb4e927f6ee admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:55.518419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb256adc-bdb4-487c-a4bf-deb4e927f6ee admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:55.537666 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:01:55.579132 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d59f1-88ac-4b9b-9663-c0284ce82c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:55.579735 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d59f1-88ac-4b9b-9663-c0284ce82c17 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:55.580010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d59f1-88ac-4b9b-9663-c0284ce82c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:55.580631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d59f1-88ac-4b9b-9663-c0284ce82c17 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:55.580931 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d59f1-88ac-4b9b-9663-c0284ce82c17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:55.582862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d59f1-88ac-4b9b-9663-c0284ce82c17 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:55.608050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1180/2360] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:01:55 2022] POST /identity/v3/users => generated 301 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:01:55.612259 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f4dc9-ca4f-473c-9a2b-447df5358027 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:55.622714 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f4dc9-ca4f-473c-9a2b-447df5358027 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:55.628869 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48eb3b-e700-44bc-b05d-260151e846c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:55.629500 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48eb3b-e700-44bc-b05d-260151e846c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:55.629776 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48eb3b-e700-44bc-b05d-260151e846c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:55.630077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48eb3b-e700-44bc-b05d-260151e846c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:55.630660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48eb3b-e700-44bc-b05d-260151e846c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:55.632259 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48eb3b-e700-44bc-b05d-260151e846c2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:55.643931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1181/2361] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:01:55.648946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d59f1-88ac-4b9b-9663-c0284ce82c17 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:55.659924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d59f1-88ac-4b9b-9663-c0284ce82c17 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:55.664566 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8d4b9-bec6-4764-be6d-808138e8d02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:55.665162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8d4b9-bec6-4764-be6d-808138e8d02d admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:55.665430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8d4b9-bec6-4764-be6d-808138e8d02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:55.665696 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8d4b9-bec6-4764-be6d-808138e8d02d admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:55.666381 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8d4b9-bec6-4764-be6d-808138e8d02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:55.668013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8d4b9-bec6-4764-be6d-808138e8d02d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:55.679052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1181/2362] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:01:55.683193 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48eb3b-e700-44bc-b05d-260151e846c2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:55.694237 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48eb3b-e700-44bc-b05d-260151e846c2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:55.698868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef7dc6-b9b4-4de5-a9e7-bc5db8286a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:55.699507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef7dc6-b9b4-4de5-a9e7-bc5db8286a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:55.699784 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef7dc6-b9b4-4de5-a9e7-bc5db8286a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:55.700056 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef7dc6-b9b4-4de5-a9e7-bc5db8286a21 admin admin] PATH_INFO: `/v3/projects/e3ccc5d0adf043d7b978d88ca22786ae/users/0d8a6994938e4d288c8388f4f2fff555/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:55.714830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef7dc6-b9b4-4de5-a9e7-bc5db8286a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3ccc5d0adf043d7b978d88ca22786ae, user_id=0d8a6994938e4d288c8388f4f2fff555, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:55.717532 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef7dc6-b9b4-4de5-a9e7-bc5db8286a21 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:55.730469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1182/2363] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:01:55 2022] PUT /identity/v3/projects/e3ccc5d0adf043d7b978d88ca22786ae/users/0d8a6994938e4d288c8388f4f2fff555/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:01:55.741979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76af684-d442-4610-91a0-80f23b3d1afe None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:55.742260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76af684-d442-4610-91a0-80f23b3d1afe None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:55.742557 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76af684-d442-4610-91a0-80f23b3d1afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:55.771343 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a76af684-d442-4610-91a0-80f23b3d1afe None None] MFA Rules not processed for user `0d8a6994938e4d288c8388f4f2fff555`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:55.772600 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a76af684-d442-4610-91a0-80f23b3d1afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:55.838300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1182/2364] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:01:55 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:01:55.843475 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef7dc6-b9b4-4de5-a9e7-bc5db8286a21 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:55.905277 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef7dc6-b9b4-4de5-a9e7-bc5db8286a21 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:55.910063 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83dc15b-534a-4262-b20a-7a1aa29ec846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:55.910707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dc15b-534a-4262-b20a-7a1aa29ec846 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:55.911001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dc15b-534a-4262-b20a-7a1aa29ec846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:55.911253 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dc15b-534a-4262-b20a-7a1aa29ec846 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:55.911868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83dc15b-534a-4262-b20a-7a1aa29ec846 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:55.913883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83dc15b-534a-4262-b20a-7a1aa29ec846 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:55.942943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1183/2365] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:01:55 2022] POST /identity/v3/projects => generated 399 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:01:55.947256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76af684-d442-4610-91a0-80f23b3d1afe None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:55.958888 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76af684-d442-4610-91a0-80f23b3d1afe None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:56.016342 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efb90b-4fec-4fb9-9004-0465854dc4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:56.016948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efb90b-4fec-4fb9-9004-0465854dc4fa admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:56.017224 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efb90b-4fec-4fb9-9004-0465854dc4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:56.017496 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efb90b-4fec-4fb9-9004-0465854dc4fa admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:56.018127 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efb90b-4fec-4fb9-9004-0465854dc4fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:56.020095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efb90b-4fec-4fb9-9004-0465854dc4fa admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:56.040609 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1183/2366] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:01:55 2022] POST /identity/v3/users => generated 306 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:01:56.044578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83dc15b-534a-4262-b20a-7a1aa29ec846 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:56.056740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83dc15b-534a-4262-b20a-7a1aa29ec846 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:56.062041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6065a1-467a-42ed-b0ba-0ac0aa7f84f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:56.062883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6065a1-467a-42ed-b0ba-0ac0aa7f84f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:56.063158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6065a1-467a-42ed-b0ba-0ac0aa7f84f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:56.063426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6065a1-467a-42ed-b0ba-0ac0aa7f84f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:56.064130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6065a1-467a-42ed-b0ba-0ac0aa7f84f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:56.065646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6065a1-467a-42ed-b0ba-0ac0aa7f84f0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:56.075943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1184/2367] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:01:56.080161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efb90b-4fec-4fb9-9004-0465854dc4fa admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:56.090613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efb90b-4fec-4fb9-9004-0465854dc4fa admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:56.095531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b49e1-d2c1-41c6-8184-26719d2fccfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:56.096213 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b49e1-d2c1-41c6-8184-26719d2fccfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:56.096509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b49e1-d2c1-41c6-8184-26719d2fccfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:56.096788 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b49e1-d2c1-41c6-8184-26719d2fccfb admin admin] PATH_INFO: `/v3/projects/542783d2eafd4f87ba47cbd2d1afcd16/users/0811c46f9ecf4f8aa180ca5d9870ac24/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:56.111776 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b49e1-d2c1-41c6-8184-26719d2fccfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=542783d2eafd4f87ba47cbd2d1afcd16, user_id=0811c46f9ecf4f8aa180ca5d9870ac24, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:56.114460 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b49e1-d2c1-41c6-8184-26719d2fccfb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:56.117446 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:01:56.129383 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1184/2368] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:01:56 2022] PUT /identity/v3/projects/542783d2eafd4f87ba47cbd2d1afcd16/users/0811c46f9ecf4f8aa180ca5d9870ac24/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:01:56.133375 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6065a1-467a-42ed-b0ba-0ac0aa7f84f0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:56.143538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6065a1-467a-42ed-b0ba-0ac0aa7f84f0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:56.151388 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:01:56.202530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371378cc-aa82-4acd-8b8c-1fa428e364d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:56.203123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371378cc-aa82-4acd-8b8c-1fa428e364d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:56.203396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371378cc-aa82-4acd-8b8c-1fa428e364d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:56.203662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371378cc-aa82-4acd-8b8c-1fa428e364d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:56.206097 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371378cc-aa82-4acd-8b8c-1fa428e364d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:56.207668 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371378cc-aa82-4acd-8b8c-1fa428e364d3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:56.217162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1185/2369] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:01:56.222163 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b49e1-d2c1-41c6-8184-26719d2fccfb admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:56.232542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b49e1-d2c1-41c6-8184-26719d2fccfb admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:56.238353 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753e396-ff05-4e9e-992e-9ff10a22e66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:56.239000 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753e396-ff05-4e9e-992e-9ff10a22e66a admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:56.239278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753e396-ff05-4e9e-992e-9ff10a22e66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:56.239573 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753e396-ff05-4e9e-992e-9ff10a22e66a admin admin] PATH_INFO: `/v3/projects/542783d2eafd4f87ba47cbd2d1afcd16/users/0811c46f9ecf4f8aa180ca5d9870ac24/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:56.241492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753e396-ff05-4e9e-992e-9ff10a22e66a admin admin] RBAC: Authorizing `identity:create_grant(project_id=542783d2eafd4f87ba47cbd2d1afcd16, user_id=0811c46f9ecf4f8aa180ca5d9870ac24, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:56.244214 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753e396-ff05-4e9e-992e-9ff10a22e66a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:56.254811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1185/2370] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:01:56 2022] PUT /identity/v3/projects/542783d2eafd4f87ba47cbd2d1afcd16/users/0811c46f9ecf4f8aa180ca5d9870ac24/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:01:56.264904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe9226-04a7-4ab1-82d6-40518a04f6eb None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:56.265186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe9226-04a7-4ab1-82d6-40518a04f6eb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:56.265458 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe9226-04a7-4ab1-82d6-40518a04f6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:56.292774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9dbe9226-04a7-4ab1-82d6-40518a04f6eb None None] MFA Rules not processed for user `0811c46f9ecf4f8aa180ca5d9870ac24`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:01:56.293960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9dbe9226-04a7-4ab1-82d6-40518a04f6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:56.362641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1186/2371] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:01:56 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:01:56.373388 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753e396-ff05-4e9e-992e-9ff10a22e66a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:56.375200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1753e396-ff05-4e9e-992e-9ff10a22e66a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:56.438777 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753e396-ff05-4e9e-992e-9ff10a22e66a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:56.442378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32449c1e-3f76-4138-8811-b816f74d8ea6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:56.442973 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32449c1e-3f76-4138-8811-b816f74d8ea6 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:56.443246 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32449c1e-3f76-4138-8811-b816f74d8ea6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:56.443511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32449c1e-3f76-4138-8811-b816f74d8ea6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:56.452260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32449c1e-3f76-4138-8811-b816f74d8ea6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:56.453776 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32449c1e-3f76-4138-8811-b816f74d8ea6 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:56.462901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1186/2372] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:01:56 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:01:56.538994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe9226-04a7-4ab1-82d6-40518a04f6eb None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:56.549561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe9226-04a7-4ab1-82d6-40518a04f6eb None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:56.553207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be790149-a6a8-4efe-b1d8-106df67dac21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:56.553801 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be790149-a6a8-4efe-b1d8-106df67dac21 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:56.554073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be790149-a6a8-4efe-b1d8-106df67dac21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:56.554340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be790149-a6a8-4efe-b1d8-106df67dac21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:56.609980 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be790149-a6a8-4efe-b1d8-106df67dac21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:56.611385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be790149-a6a8-4efe-b1d8-106df67dac21 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:56.620000 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1187/2373] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:01:56 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:01:59.251922 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32449c1e-3f76-4138-8811-b816f74d8ea6 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:01:59.253376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-32449c1e-3f76-4138-8811-b816f74d8ea6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:59.310862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32449c1e-3f76-4138-8811-b816f74d8ea6 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:01:59.315267 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b7984-d718-44ea-9c7c-e755f396d4a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:01:59.315868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b7984-d718-44ea-9c7c-e755f396d4a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:01:59.316176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b7984-d718-44ea-9c7c-e755f396d4a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:01:59.316450 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b7984-d718-44ea-9c7c-e755f396d4a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:01:59.318165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0a2b7984-d718-44ea-9c7c-e755f396d4a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:01:59.384187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b7984-d718-44ea-9c7c-e755f396d4a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:01:59.385810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b7984-d718-44ea-9c7c-e755f396d4a0 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:01:59.395924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1187/2374] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:00.736459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be790149-a6a8-4efe-b1d8-106df67dac21 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:00.747972 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be790149-a6a8-4efe-b1d8-106df67dac21 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:00.754396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493db6e-5e23-46fb-95c8-6e2cfc9bd8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:00.755058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493db6e-5e23-46fb-95c8-6e2cfc9bd8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:00.755351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493db6e-5e23-46fb-95c8-6e2cfc9bd8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:00.755627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493db6e-5e23-46fb-95c8-6e2cfc9bd8cf admin admin] PATH_INFO: `/v3/users/0d8a6994938e4d288c8388f4f2fff555` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:00.756786 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493db6e-5e23-46fb-95c8-6e2cfc9bd8cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d8a6994938e4d288c8388f4f2fff555)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:00.758859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493db6e-5e23-46fb-95c8-6e2cfc9bd8cf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:00.791276 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:00.816736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1493db6e-5e23-46fb-95c8-6e2cfc9bd8cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d8a6994938e4d288c8388f4f2fff555'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:00.831237 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1493db6e-5e23-46fb-95c8-6e2cfc9bd8cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d8a6994938e4d288c8388f4f2fff555'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:00.839319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1493db6e-5e23-46fb-95c8-6e2cfc9bd8cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d8a6994938e4d288c8388f4f2fff555'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:00.840306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1493db6e-5e23-46fb-95c8-6e2cfc9bd8cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d8a6994938e4d288c8388f4f2fff555'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:00.849906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1493db6e-5e23-46fb-95c8-6e2cfc9bd8cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d8a6994938e4d288c8388f4f2fff555'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:00.861417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1188/2375] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:00 2022] DELETE /identity/v3/users/0d8a6994938e4d288c8388f4f2fff555 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:00.863103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8fcc9-5791-450c-af4c-1ce0992856c2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:00.863616 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8fcc9-5791-450c-af4c-1ce0992856c2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:00.863897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8fcc9-5791-450c-af4c-1ce0992856c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:00.891713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-fda8fcc9-5791-450c-af4c-1ce0992856c2 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:00.893225 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fda8fcc9-5791-450c-af4c-1ce0992856c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:00.941940 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:00.983946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1188/2376] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:00.994111 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493db6e-5e23-46fb-95c8-6e2cfc9bd8cf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:00.995543 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1493db6e-5e23-46fb-95c8-6e2cfc9bd8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:01.085384 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493db6e-5e23-46fb-95c8-6e2cfc9bd8cf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:01.088929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83393dc7-8d81-4dd9-8f34-bf910c9e8e43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:01.089539 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83393dc7-8d81-4dd9-8f34-bf910c9e8e43 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:01.089813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83393dc7-8d81-4dd9-8f34-bf910c9e8e43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:01.090078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83393dc7-8d81-4dd9-8f34-bf910c9e8e43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:01.100474 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83393dc7-8d81-4dd9-8f34-bf910c9e8e43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:01.101900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83393dc7-8d81-4dd9-8f34-bf910c9e8e43 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:01.113392 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1189/2377] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:01.423326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8fcc9-5791-450c-af4c-1ce0992856c2 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:01.424883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fda8fcc9-5791-450c-af4c-1ce0992856c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:01.441007 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8fcc9-5791-450c-af4c-1ce0992856c2 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:01.449366 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:01.450246 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:01.450707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:01.450980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781 admin admin] PATH_INFO: `/v3/projects/e3ccc5d0adf043d7b978d88ca22786ae` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:01.453321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3ccc5d0adf043d7b978d88ca22786ae)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:01.455453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:01.506294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3ccc5d0adf043d7b978d88ca22786ae', 'request_id': 'req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:01.511844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3ccc5d0adf043d7b978d88ca22786ae', 'request_id': 'req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:01.518753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3ccc5d0adf043d7b978d88ca22786ae', 'request_id': 'req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:01.531337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781 admin admin] The token cache is being invalidate because project e3ccc5d0adf043d7b978d88ca22786ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:01.531635 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:01.532331 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:01.533864 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1189/2378] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:01 2022] DELETE /identity/v3/projects/e3ccc5d0adf043d7b978d88ca22786ae => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:01.537865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83393dc7-8d81-4dd9-8f34-bf910c9e8e43 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:01.547968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83393dc7-8d81-4dd9-8f34-bf910c9e8e43 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:01.607160 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd97e72-8b9b-464c-b686-354291903c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:01.607899 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd97e72-8b9b-464c-b686-354291903c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:01.608317 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd97e72-8b9b-464c-b686-354291903c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:01.608769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd97e72-8b9b-464c-b686-354291903c42 admin admin] PATH_INFO: `/v3/users/0811c46f9ecf4f8aa180ca5d9870ac24` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:01.610384 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd97e72-8b9b-464c-b686-354291903c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0811c46f9ecf4f8aa180ca5d9870ac24)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:01.612979 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd97e72-8b9b-464c-b686-354291903c42 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:01.672295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0dd97e72-8b9b-464c-b686-354291903c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0811c46f9ecf4f8aa180ca5d9870ac24'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:01.673830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:01.693307 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0dd97e72-8b9b-464c-b686-354291903c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0811c46f9ecf4f8aa180ca5d9870ac24'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:01.703825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0dd97e72-8b9b-464c-b686-354291903c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0811c46f9ecf4f8aa180ca5d9870ac24'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:01.704695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0dd97e72-8b9b-464c-b686-354291903c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0811c46f9ecf4f8aa180ca5d9870ac24'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:01.716701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0dd97e72-8b9b-464c-b686-354291903c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0811c46f9ecf4f8aa180ca5d9870ac24'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:01.723341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1190/2379] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:01 2022] DELETE /identity/v3/users/0811c46f9ecf4f8aa180ca5d9870ac24 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:02.203745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:02.206031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:02.221174 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd8e6f-ca1b-41f7-b9b2-06cea4b2c781 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:02.300053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e52d7-2919-4093-bfd3-18ae288f2e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:02.300053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e52d7-2919-4093-bfd3-18ae288f2e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:02.300053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e52d7-2919-4093-bfd3-18ae288f2e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:02.300053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e52d7-2919-4093-bfd3-18ae288f2e0d admin admin] PATH_INFO: `/v3/projects/542783d2eafd4f87ba47cbd2d1afcd16` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:02.301671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e52d7-2919-4093-bfd3-18ae288f2e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=542783d2eafd4f87ba47cbd2d1afcd16)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:02.304166 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e52d7-2919-4093-bfd3-18ae288f2e0d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:02.364765 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f99e52d7-2919-4093-bfd3-18ae288f2e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '542783d2eafd4f87ba47cbd2d1afcd16', 'request_id': 'req-f99e52d7-2919-4093-bfd3-18ae288f2e0d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:02.372663 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f99e52d7-2919-4093-bfd3-18ae288f2e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '542783d2eafd4f87ba47cbd2d1afcd16', 'request_id': 'req-f99e52d7-2919-4093-bfd3-18ae288f2e0d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:02.376482 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:02.386872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f99e52d7-2919-4093-bfd3-18ae288f2e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '542783d2eafd4f87ba47cbd2d1afcd16', 'request_id': 'req-f99e52d7-2919-4093-bfd3-18ae288f2e0d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:02.407483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f99e52d7-2919-4093-bfd3-18ae288f2e0d admin admin] The token cache is being invalidate because project 542783d2eafd4f87ba47cbd2d1afcd16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:02.407919 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f99e52d7-2919-4093-bfd3-18ae288f2e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:02.409617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f99e52d7-2919-4093-bfd3-18ae288f2e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:02.415288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1190/2380] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:02 2022] DELETE /identity/v3/projects/542783d2eafd4f87ba47cbd2d1afcd16 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:02.417119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13caeaf3-0b67-4923-b6bb-d49c225c4179 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:02.418056 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13caeaf3-0b67-4923-b6bb-d49c225c4179 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:02.418528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13caeaf3-0b67-4923-b6bb-d49c225c4179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:02.444892 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-13caeaf3-0b67-4923-b6bb-d49c225c4179 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:02.446145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-13caeaf3-0b67-4923-b6bb-d49c225c4179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:02.541645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1191/2381] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:02.553332 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9876c5-2266-47d8-b911-a20dc81c6606 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:02.553620 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9876c5-2266-47d8-b911-a20dc81c6606 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:02.553896 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9876c5-2266-47d8-b911-a20dc81c6606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:02.557281 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13caeaf3-0b67-4923-b6bb-d49c225c4179 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:02.558765 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-13caeaf3-0b67-4923-b6bb-d49c225c4179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:02.576580 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7a9876c5-2266-47d8-b911-a20dc81c6606 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:02.577940 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7a9876c5-2266-47d8-b911-a20dc81c6606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:02.589628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1191/2382] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:02.590600 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9876c5-2266-47d8-b911-a20dc81c6606 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:02.610264 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9876c5-2266-47d8-b911-a20dc81c6606 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:02.615541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2645c0f-4258-49ad-b857-5118b7e448e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:02.616386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2645c0f-4258-49ad-b857-5118b7e448e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:02.616682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2645c0f-4258-49ad-b857-5118b7e448e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:02.617098 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2645c0f-4258-49ad-b857-5118b7e448e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:02.617746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2645c0f-4258-49ad-b857-5118b7e448e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:02:02.618114 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2645c0f-4258-49ad-b857-5118b7e448e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:02.619796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2645c0f-4258-49ad-b857-5118b7e448e0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:02.634202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1192/2383] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 08:02:02.648268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2645c0f-4258-49ad-b857-5118b7e448e0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:02.653010 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13caeaf3-0b67-4923-b6bb-d49c225c4179 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:02.658597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae9e62-c1b9-42aa-be43-928068bd3139 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:02.659248 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae9e62-c1b9-42aa-be43-928068bd3139 service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:02.659535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae9e62-c1b9-42aa-be43-928068bd3139 service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:02.659809 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae9e62-c1b9-42aa-be43-928068bd3139 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:02.661561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e5ae9e62-c1b9-42aa-be43-928068bd3139 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:02.661738 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2645c0f-4258-49ad-b857-5118b7e448e0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:02.666531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a37e3-0e91-47cc-a070-c6058c2fbe9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:02.667128 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a37e3-0e91-47cc-a070-c6058c2fbe9b admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:02.667524 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a37e3-0e91-47cc-a070-c6058c2fbe9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:02.667803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a37e3-0e91-47cc-a070-c6058c2fbe9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:02.668535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a37e3-0e91-47cc-a070-c6058c2fbe9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:02.671014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a37e3-0e91-47cc-a070-c6058c2fbe9b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:02.673856 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae9e62-c1b9-42aa-be43-928068bd3139 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:02.675412 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae9e62-c1b9-42aa-be43-928068bd3139 service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:02.685451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1192/2384] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:02:02.686853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae9e62-c1b9-42aa-be43-928068bd3139 service placement] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:02.688397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e5ae9e62-c1b9-42aa-be43-928068bd3139 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:02.706971 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1193/2385] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:02 2022] POST /identity/v3/projects => generated 407 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:02.707124 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae9e62-c1b9-42aa-be43-928068bd3139 service placement] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:02.707847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a37e3-0e91-47cc-a070-c6058c2fbe9b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:02.758852 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:02.767565 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3cff4-3a1d-4e29-a56a-82a49f9c04c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:02.768496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3cff4-3a1d-4e29-a56a-82a49f9c04c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:02.768970 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3cff4-3a1d-4e29-a56a-82a49f9c04c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:02.769415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3cff4-3a1d-4e29-a56a-82a49f9c04c4 admin admin] PATH_INFO: `/v3/users/e07bc40275734f628fcc2f62dd4dd0e0` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:02.770579 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3cff4-3a1d-4e29-a56a-82a49f9c04c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e07bc40275734f628fcc2f62dd4dd0e0)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:02.772562 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3cff4-3a1d-4e29-a56a-82a49f9c04c4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:02.772690 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a37e3-0e91-47cc-a070-c6058c2fbe9b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:02.779075 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e575b-2303-4cda-be7e-cc0f29ef34ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:02.779773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e575b-2303-4cda-be7e-cc0f29ef34ab admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:02.780049 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e575b-2303-4cda-be7e-cc0f29ef34ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:02.780375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e575b-2303-4cda-be7e-cc0f29ef34ab admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:02.780977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e575b-2303-4cda-be7e-cc0f29ef34ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:02.783343 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e575b-2303-4cda-be7e-cc0f29ef34ab admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:02.807595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1194/2386] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:02 2022] POST /identity/v3/users => generated 304 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:02.812558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e575b-2303-4cda-be7e-cc0f29ef34ab admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:02.819871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-57f3cff4-3a1d-4e29-a56a-82a49f9c04c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e07bc40275734f628fcc2f62dd4dd0e0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:02.822209 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e575b-2303-4cda-be7e-cc0f29ef34ab admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:02.826985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e08d9-8870-4597-9af7-eeba93baeee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:02.827571 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e08d9-8870-4597-9af7-eeba93baeee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:02.827851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e08d9-8870-4597-9af7-eeba93baeee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:02.828157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e08d9-8870-4597-9af7-eeba93baeee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:02.828790 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e08d9-8870-4597-9af7-eeba93baeee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:02.830384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e08d9-8870-4597-9af7-eeba93baeee1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:02.831956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-57f3cff4-3a1d-4e29-a56a-82a49f9c04c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e07bc40275734f628fcc2f62dd4dd0e0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:02.840549 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1195/2387] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:02.841181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-57f3cff4-3a1d-4e29-a56a-82a49f9c04c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e07bc40275734f628fcc2f62dd4dd0e0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:02.841836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-57f3cff4-3a1d-4e29-a56a-82a49f9c04c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e07bc40275734f628fcc2f62dd4dd0e0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:02.844832 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e08d9-8870-4597-9af7-eeba93baeee1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:02.849620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-57f3cff4-3a1d-4e29-a56a-82a49f9c04c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e07bc40275734f628fcc2f62dd4dd0e0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:02.855971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1193/2388] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:02 2022] DELETE /identity/v3/users/e07bc40275734f628fcc2f62dd4dd0e0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:02.859468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e08d9-8870-4597-9af7-eeba93baeee1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:02.916989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de1257-a651-44f7-b2f6-cc9d56f5f219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:02.917598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de1257-a651-44f7-b2f6-cc9d56f5f219 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:02.917874 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de1257-a651-44f7-b2f6-cc9d56f5f219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:02.918143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de1257-a651-44f7-b2f6-cc9d56f5f219 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:02.918770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de1257-a651-44f7-b2f6-cc9d56f5f219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:02.920925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de1257-a651-44f7-b2f6-cc9d56f5f219 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:02.931012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1196/2389] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:02.935445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3cff4-3a1d-4e29-a56a-82a49f9c04c4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:02.937994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-57f3cff4-3a1d-4e29-a56a-82a49f9c04c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:02.951066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3cff4-3a1d-4e29-a56a-82a49f9c04c4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:02.956239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92283c65-f8b9-4a2f-b919-d55b72bb32d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:02.956931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92283c65-f8b9-4a2f-b919-d55b72bb32d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:02.957229 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92283c65-f8b9-4a2f-b919-d55b72bb32d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:02.957501 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92283c65-f8b9-4a2f-b919-d55b72bb32d7 admin admin] PATH_INFO: `/v3/projects/c1bbaaa513b64fbe9931dcd70816d965/users/3ab76e7a5671435ebf3bfe817a7e4361/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:02.973712 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92283c65-f8b9-4a2f-b919-d55b72bb32d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1bbaaa513b64fbe9931dcd70816d965, user_id=3ab76e7a5671435ebf3bfe817a7e4361, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:02.976484 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92283c65-f8b9-4a2f-b919-d55b72bb32d7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:02.990099 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1194/2390] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:02 2022] PUT /identity/v3/projects/c1bbaaa513b64fbe9931dcd70816d965/users/3ab76e7a5671435ebf3bfe817a7e4361/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:03.000680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e88cf-dbe6-4748-a5a7-799192fe3682 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.000963 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e88cf-dbe6-4748-a5a7-799192fe3682 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.001232 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e88cf-dbe6-4748-a5a7-799192fe3682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.033001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92283c65-f8b9-4a2f-b919-d55b72bb32d7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:03.035058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-92283c65-f8b9-4a2f-b919-d55b72bb32d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:03.035311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-034e88cf-dbe6-4748-a5a7-799192fe3682 None None] MFA Rules not processed for user `3ab76e7a5671435ebf3bfe817a7e4361`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:03.036588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-034e88cf-dbe6-4748-a5a7-799192fe3682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:03.049152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:03.114993 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1197/2391] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:02 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:03.119461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e88cf-dbe6-4748-a5a7-799192fe3682 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:03.119595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92283c65-f8b9-4a2f-b919-d55b72bb32d7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:03.124878 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c683e87-55e4-47b9-b396-86625efee9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:03.125505 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c683e87-55e4-47b9-b396-86625efee9c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.125781 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c683e87-55e4-47b9-b396-86625efee9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.126052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c683e87-55e4-47b9-b396-86625efee9c2 admin admin] PATH_INFO: `/v3/projects/09a6411dd18c4b6a8fad228058c8e06b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.127175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c683e87-55e4-47b9-b396-86625efee9c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09a6411dd18c4b6a8fad228058c8e06b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:03.129128 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c683e87-55e4-47b9-b396-86625efee9c2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:03.130444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e88cf-dbe6-4748-a5a7-799192fe3682 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:03.135518 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffa2af-5452-464c-bc38-555e5c1f52df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:03.136199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffa2af-5452-464c-bc38-555e5c1f52df admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.136674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffa2af-5452-464c-bc38-555e5c1f52df admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.136953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffa2af-5452-464c-bc38-555e5c1f52df admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.137580 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffa2af-5452-464c-bc38-555e5c1f52df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:03.139806 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffa2af-5452-464c-bc38-555e5c1f52df admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:03.168656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1198/2392] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:03 2022] POST /identity/v3/projects => generated 407 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:03.173546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffa2af-5452-464c-bc38-555e5c1f52df admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:03.181602 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5c683e87-55e4-47b9-b396-86625efee9c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09a6411dd18c4b6a8fad228058c8e06b', 'request_id': 'req-5c683e87-55e4-47b9-b396-86625efee9c2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:03.183659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:03.188976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5c683e87-55e4-47b9-b396-86625efee9c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09a6411dd18c4b6a8fad228058c8e06b', 'request_id': 'req-5c683e87-55e4-47b9-b396-86625efee9c2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:03.197034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5c683e87-55e4-47b9-b396-86625efee9c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09a6411dd18c4b6a8fad228058c8e06b', 'request_id': 'req-5c683e87-55e4-47b9-b396-86625efee9c2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:03.210035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5c683e87-55e4-47b9-b396-86625efee9c2 admin admin] The token cache is being invalidate because project 09a6411dd18c4b6a8fad228058c8e06b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:03.210324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5c683e87-55e4-47b9-b396-86625efee9c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:03.210957 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5c683e87-55e4-47b9-b396-86625efee9c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:03.215883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1195/2393] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:03 2022] DELETE /identity/v3/projects/09a6411dd18c4b6a8fad228058c8e06b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:03.217859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91613ea3-dc14-4727-b9eb-597ebef46c45 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.218155 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91613ea3-dc14-4727-b9eb-597ebef46c45 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.218427 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91613ea3-dc14-4727-b9eb-597ebef46c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.239499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-91613ea3-dc14-4727-b9eb-597ebef46c45 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:03.240538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffa2af-5452-464c-bc38-555e5c1f52df admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:03.240988 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-91613ea3-dc14-4727-b9eb-597ebef46c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:03.245378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f6f90-81c5-422c-a5d8-0901e9c006fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:03.246052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f6f90-81c5-422c-a5d8-0901e9c006fb admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.246399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f6f90-81c5-422c-a5d8-0901e9c006fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.246748 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f6f90-81c5-422c-a5d8-0901e9c006fb admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.247423 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f6f90-81c5-422c-a5d8-0901e9c006fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:03.248696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1196/2394] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:03.249838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f6f90-81c5-422c-a5d8-0901e9c006fb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:03.256366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c515d7-9114-4a0e-be62-9ddd6bae4259 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.256649 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c515d7-9114-4a0e-be62-9ddd6bae4259 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.256921 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c515d7-9114-4a0e-be62-9ddd6bae4259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.274617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1199/2395] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:03 2022] POST /identity/v3/users => generated 310 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:03.275690 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f6f90-81c5-422c-a5d8-0901e9c006fb admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:03.276946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-68c515d7-9114-4a0e-be62-9ddd6bae4259 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:03.277335 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cc3f6f90-81c5-422c-a5d8-0901e9c006fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:03.278113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-68c515d7-9114-4a0e-be62-9ddd6bae4259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:03.287429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1197/2396] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:03.289787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f6f90-81c5-422c-a5d8-0901e9c006fb admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:03.291361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c515d7-9114-4a0e-be62-9ddd6bae4259 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:03.294344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f16837-b937-4ed2-93ce-0df540453219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:03.294925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f16837-b937-4ed2-93ce-0df540453219 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.295821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f16837-b937-4ed2-93ce-0df540453219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.296121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f16837-b937-4ed2-93ce-0df540453219 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.296935 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f16837-b937-4ed2-93ce-0df540453219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:03.298528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f16837-b937-4ed2-93ce-0df540453219 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:03.301146 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c515d7-9114-4a0e-be62-9ddd6bae4259 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:03.305737 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2290e81-f502-4257-a0a5-f99f0a3695f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:03.306314 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2290e81-f502-4257-a0a5-f99f0a3695f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.306587 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2290e81-f502-4257-a0a5-f99f0a3695f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.306886 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2290e81-f502-4257-a0a5-f99f0a3695f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.307519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2290e81-f502-4257-a0a5-f99f0a3695f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:02:03.307818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2290e81-f502-4257-a0a5-f99f0a3695f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:03.308369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1200/2397] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:03.313567 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f16837-b937-4ed2-93ce-0df540453219 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:03.313701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2290e81-f502-4257-a0a5-f99f0a3695f4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:03.323543 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f16837-b937-4ed2-93ce-0df540453219 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:03.329656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1198/2398] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 08:02:03.329820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a544cd5-62f7-48d6-8f8a-9ca28a273a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:03.329820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a544cd5-62f7-48d6-8f8a-9ca28a273a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.330061 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a544cd5-62f7-48d6-8f8a-9ca28a273a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.330330 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a544cd5-62f7-48d6-8f8a-9ca28a273a04 admin admin] PATH_INFO: `/v3/projects/0b6c697204804a569f78d629f566dbcd/users/68c30a51c9cd485faacfb5da289a7bcd/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.330866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2290e81-f502-4257-a0a5-f99f0a3695f4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:03.340520 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2290e81-f502-4257-a0a5-f99f0a3695f4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:03.345470 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f62377-7a2a-4b08-bff2-5be05ecfbace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:03.345995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a544cd5-62f7-48d6-8f8a-9ca28a273a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6c697204804a569f78d629f566dbcd, user_id=68c30a51c9cd485faacfb5da289a7bcd, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:03.346126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f62377-7a2a-4b08-bff2-5be05ecfbace admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.346310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f62377-7a2a-4b08-bff2-5be05ecfbace admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.346576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f62377-7a2a-4b08-bff2-5be05ecfbace admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.347189 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f62377-7a2a-4b08-bff2-5be05ecfbace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:03.348784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a544cd5-62f7-48d6-8f8a-9ca28a273a04 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:03.349068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f62377-7a2a-4b08-bff2-5be05ecfbace admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:03.363439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1201/2399] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:03 2022] PUT /identity/v3/projects/0b6c697204804a569f78d629f566dbcd/users/68c30a51c9cd485faacfb5da289a7bcd/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:03.367541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a544cd5-62f7-48d6-8f8a-9ca28a273a04 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:03.380111 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1199/2400] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:03 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:03.381922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f62377-7a2a-4b08-bff2-5be05ecfbace admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:03.418364 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:03.434611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a544cd5-62f7-48d6-8f8a-9ca28a273a04 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:03.448964 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f62377-7a2a-4b08-bff2-5be05ecfbace admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:03.454398 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343e3bb-9056-4b60-a768-7ac220ade511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:03.455005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343e3bb-9056-4b60-a768-7ac220ade511 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.455280 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343e3bb-9056-4b60-a768-7ac220ade511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.455552 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343e3bb-9056-4b60-a768-7ac220ade511 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.456197 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343e3bb-9056-4b60-a768-7ac220ade511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:03.458112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343e3bb-9056-4b60-a768-7ac220ade511 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:03.489567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1200/2401] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:03 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:03.493660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343e3bb-9056-4b60-a768-7ac220ade511 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:03.494559 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25a532-296d-4355-afef-eed9afd790f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:03.495165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25a532-296d-4355-afef-eed9afd790f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.495439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25a532-296d-4355-afef-eed9afd790f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.495707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25a532-296d-4355-afef-eed9afd790f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.496360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25a532-296d-4355-afef-eed9afd790f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:03.497943 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25a532-296d-4355-afef-eed9afd790f3 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:03.504396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343e3bb-9056-4b60-a768-7ac220ade511 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:03.511740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1202/2402] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:03.511875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce03a6-a9dd-4c46-8351-888952aeb90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:03.511875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce03a6-a9dd-4c46-8351-888952aeb90b admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.511875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce03a6-a9dd-4c46-8351-888952aeb90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.511875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce03a6-a9dd-4c46-8351-888952aeb90b admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.512277 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce03a6-a9dd-4c46-8351-888952aeb90b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:03.512536 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce03a6-a9dd-4c46-8351-888952aeb90b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:03.512708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25a532-296d-4355-afef-eed9afd790f3 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:03.522313 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25a532-296d-4355-afef-eed9afd790f3 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:03.525844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1201/2403] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:03.526806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce03a6-a9dd-4c46-8351-888952aeb90b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:03.527258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e6385-c780-4547-94e1-0b66c53bc61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:03.527897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e6385-c780-4547-94e1-0b66c53bc61b admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.528201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e6385-c780-4547-94e1-0b66c53bc61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.528470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e6385-c780-4547-94e1-0b66c53bc61b admin admin] PATH_INFO: `/v3/projects/0b6c697204804a569f78d629f566dbcd/users/68c30a51c9cd485faacfb5da289a7bcd/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.530640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e6385-c780-4547-94e1-0b66c53bc61b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6c697204804a569f78d629f566dbcd, user_id=68c30a51c9cd485faacfb5da289a7bcd, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:03.534625 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e6385-c780-4547-94e1-0b66c53bc61b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:03.546219 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce03a6-a9dd-4c46-8351-888952aeb90b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:03.549891 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1203/2404] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:03 2022] PUT /identity/v3/projects/0b6c697204804a569f78d629f566dbcd/users/68c30a51c9cd485faacfb5da289a7bcd/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:03.560139 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8316b-c48a-4fb0-a9d7-9dc07e8f12ee None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.560424 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8316b-c48a-4fb0-a9d7-9dc07e8f12ee None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.560697 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8316b-c48a-4fb0-a9d7-9dc07e8f12ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.589314 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-bab8316b-c48a-4fb0-a9d7-9dc07e8f12ee None None] MFA Rules not processed for user `68c30a51c9cd485faacfb5da289a7bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:03.590274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-bab8316b-c48a-4fb0-a9d7-9dc07e8f12ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:03.605011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4df58-3dfd-49a4-a903-d4009cb805da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:03.605613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4df58-3dfd-49a4-a903-d4009cb805da admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.605891 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4df58-3dfd-49a4-a903-d4009cb805da admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.606163 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4df58-3dfd-49a4-a903-d4009cb805da admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.606826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4df58-3dfd-49a4-a903-d4009cb805da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:03.609059 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4df58-3dfd-49a4-a903-d4009cb805da admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:03.619812 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1202/2405] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:03.624472 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4df58-3dfd-49a4-a903-d4009cb805da admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:03.635295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4df58-3dfd-49a4-a903-d4009cb805da admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:03.641005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239424d-b63d-429b-bab7-7f263bf28c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:03.642337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239424d-b63d-429b-bab7-7f263bf28c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.642637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239424d-b63d-429b-bab7-7f263bf28c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.642912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239424d-b63d-429b-bab7-7f263bf28c66 admin admin] PATH_INFO: `/v3/projects/fd9519de9f474cfabafdb669b7e76269/users/a4a54d9238414d4aa2c6777b0dbca15e/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.646524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:03.661256 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7239424d-b63d-429b-bab7-7f263bf28c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9519de9f474cfabafdb669b7e76269, user_id=a4a54d9238414d4aa2c6777b0dbca15e, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:03.663904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7239424d-b63d-429b-bab7-7f263bf28c66 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:03.665565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1204/2406] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:03 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:03.677003 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8316b-c48a-4fb0-a9d7-9dc07e8f12ee None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:03.677680 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1203/2407] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:03 2022] PUT /identity/v3/projects/fd9519de9f474cfabafdb669b7e76269/users/a4a54d9238414d4aa2c6777b0dbca15e/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:03.680968 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-bab8316b-c48a-4fb0-a9d7-9dc07e8f12ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:03.691003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fe539-b043-4d29-b1d2-59d86f6b8a1b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.691213 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fe539-b043-4d29-b1d2-59d86f6b8a1b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.691486 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fe539-b043-4d29-b1d2-59d86f6b8a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.718755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b43fe539-b043-4d29-b1d2-59d86f6b8a1b None None] MFA Rules not processed for user `a4a54d9238414d4aa2c6777b0dbca15e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:03.719963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b43fe539-b043-4d29-b1d2-59d86f6b8a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:03.740546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8316b-c48a-4fb0-a9d7-9dc07e8f12ee None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:03.752064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:03.786867 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1204/2408] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:03.791249 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fe539-b043-4d29-b1d2-59d86f6b8a1b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:03.794015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11371a-61a6-469b-8ae2-56b52122dfc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:03.794697 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11371a-61a6-469b-8ae2-56b52122dfc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.795154 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11371a-61a6-469b-8ae2-56b52122dfc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.796315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11371a-61a6-469b-8ae2-56b52122dfc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.854077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fe539-b043-4d29-b1d2-59d86f6b8a1b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:03.855492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11371a-61a6-469b-8ae2-56b52122dfc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:03.856967 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11371a-61a6-469b-8ae2-56b52122dfc8 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:03.859362 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed9db3-d008-4637-98bf-2134e557f17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:03.859959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed9db3-d008-4637-98bf-2134e557f17c admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.860269 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed9db3-d008-4637-98bf-2134e557f17c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.860544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed9db3-d008-4637-98bf-2134e557f17c admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.861175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed9db3-d008-4637-98bf-2134e557f17c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:03.863058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed9db3-d008-4637-98bf-2134e557f17c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:03.867422 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1205/2409] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:03 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:02:03.893471 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1205/2410] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:03 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:03.897616 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11371a-61a6-469b-8ae2-56b52122dfc8 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:03.908150 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11371a-61a6-469b-8ae2-56b52122dfc8 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:03.970772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947203e-39da-4f54-b20e-8c4ae601f014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:03.971330 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947203e-39da-4f54-b20e-8c4ae601f014 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:03.971612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947203e-39da-4f54-b20e-8c4ae601f014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:03.971936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947203e-39da-4f54-b20e-8c4ae601f014 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:03.972594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947203e-39da-4f54-b20e-8c4ae601f014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:03.974650 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947203e-39da-4f54-b20e-8c4ae601f014 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:03.996115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1206/2411] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:03 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:04.000914 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed9db3-d008-4637-98bf-2134e557f17c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:04.011014 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed9db3-d008-4637-98bf-2134e557f17c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:04.016808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed6113-379b-4c83-b20f-f82eabf9cc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:04.017414 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed6113-379b-4c83-b20f-f82eabf9cc55 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:04.017692 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed6113-379b-4c83-b20f-f82eabf9cc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:04.017965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed6113-379b-4c83-b20f-f82eabf9cc55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:04.018732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed6113-379b-4c83-b20f-f82eabf9cc55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:04.021759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed6113-379b-4c83-b20f-f82eabf9cc55 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:04.032288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947203e-39da-4f54-b20e-8c4ae601f014 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:04.034055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1206/2412] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:04.038065 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed6113-379b-4c83-b20f-f82eabf9cc55 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:04.049291 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed6113-379b-4c83-b20f-f82eabf9cc55 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:04.054120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5d8a4-6caa-4994-8779-67088fac25f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:04.054765 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5d8a4-6caa-4994-8779-67088fac25f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:04.055045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5d8a4-6caa-4994-8779-67088fac25f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:04.055344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5d8a4-6caa-4994-8779-67088fac25f5 admin admin] PATH_INFO: `/v3/projects/c4cfce19806d4f7ea1ba06e8c21788a0/users/8b83e3c8c834413d85c0c5e868947fd9/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:04.071168 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5d8a4-6caa-4994-8779-67088fac25f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4cfce19806d4f7ea1ba06e8c21788a0, user_id=8b83e3c8c834413d85c0c5e868947fd9, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:04.073844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5d8a4-6caa-4994-8779-67088fac25f5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:04.087962 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1207/2413] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:04 2022] PUT /identity/v3/projects/c4cfce19806d4f7ea1ba06e8c21788a0/users/8b83e3c8c834413d85c0c5e868947fd9/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:04.091324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947203e-39da-4f54-b20e-8c4ae601f014 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:04.092605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5d8a4-6caa-4994-8779-67088fac25f5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:04.101234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:04.121455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:04.147105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00be880-1052-4a9b-98e1-2f525b3d108b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:04.147736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00be880-1052-4a9b-98e1-2f525b3d108b service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:04.148014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00be880-1052-4a9b-98e1-2f525b3d108b service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:04.148314 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00be880-1052-4a9b-98e1-2f525b3d108b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:04.150099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d00be880-1052-4a9b-98e1-2f525b3d108b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:04.160044 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5d8a4-6caa-4994-8779-67088fac25f5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:04.164972 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7d18b-709a-49ac-8073-9cdeea086d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:04.165579 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7d18b-709a-49ac-8073-9cdeea086d26 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:04.165859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7d18b-709a-49ac-8073-9cdeea086d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:04.166131 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7d18b-709a-49ac-8073-9cdeea086d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:04.166768 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7d18b-709a-49ac-8073-9cdeea086d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:04.168377 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7d18b-709a-49ac-8073-9cdeea086d26 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:04.179655 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1208/2414] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:04.183915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7d18b-709a-49ac-8073-9cdeea086d26 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:04.193992 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7d18b-709a-49ac-8073-9cdeea086d26 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:04.199418 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f032a6bf-a2ec-4272-8c10-64ef591014e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:04.200044 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032a6bf-a2ec-4272-8c10-64ef591014e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:04.200352 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032a6bf-a2ec-4272-8c10-64ef591014e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:04.200625 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032a6bf-a2ec-4272-8c10-64ef591014e8 admin admin] PATH_INFO: `/v3/projects/c4cfce19806d4f7ea1ba06e8c21788a0/users/8b83e3c8c834413d85c0c5e868947fd9/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:04.203181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f032a6bf-a2ec-4272-8c10-64ef591014e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4cfce19806d4f7ea1ba06e8c21788a0, user_id=8b83e3c8c834413d85c0c5e868947fd9, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:04.205866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f032a6bf-a2ec-4272-8c10-64ef591014e8 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:04.210876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00be880-1052-4a9b-98e1-2f525b3d108b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:04.212345 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00be880-1052-4a9b-98e1-2f525b3d108b service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:04.216064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1209/2415] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:04 2022] PUT /identity/v3/projects/c4cfce19806d4f7ea1ba06e8c21788a0/users/8b83e3c8c834413d85c0c5e868947fd9/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:04.224310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1207/2416] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:04 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:02:04.227162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed55426-d016-4114-a925-6a51a0b2f5d9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:04.227445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed55426-d016-4114-a925-6a51a0b2f5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:04.227776 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed55426-d016-4114-a925-6a51a0b2f5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:04.256546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5ed55426-d016-4114-a925-6a51a0b2f5d9 None None] MFA Rules not processed for user `8b83e3c8c834413d85c0c5e868947fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:04.257746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5ed55426-d016-4114-a925-6a51a0b2f5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:04.330263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1210/2417] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:04.342558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00be880-1052-4a9b-98e1-2f525b3d108b service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:04.344346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d00be880-1052-4a9b-98e1-2f525b3d108b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:04.355513 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00be880-1052-4a9b-98e1-2f525b3d108b service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:04.412299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b9b3f-26be-4937-a79d-8a08a7479ff3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:04.412917 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b9b3f-26be-4937-a79d-8a08a7479ff3 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:04.413198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b9b3f-26be-4937-a79d-8a08a7479ff3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:04.413472 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b9b3f-26be-4937-a79d-8a08a7479ff3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:04.425663 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b9b3f-26be-4937-a79d-8a08a7479ff3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:04.427128 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b9b3f-26be-4937-a79d-8a08a7479ff3 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:04.439609 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1208/2418] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:04.861192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed55426-d016-4114-a925-6a51a0b2f5d9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:04.928823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed55426-d016-4114-a925-6a51a0b2f5d9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:04.933259 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1c5df-8cdb-4df9-b76b-83031dcdb7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:04.933857 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1c5df-8cdb-4df9-b76b-83031dcdb7df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:04.934128 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1c5df-8cdb-4df9-b76b-83031dcdb7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:04.934395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1c5df-8cdb-4df9-b76b-83031dcdb7df admin admin] PATH_INFO: `/v3/users/a4a54d9238414d4aa2c6777b0dbca15e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:04.935614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1c5df-8cdb-4df9-b76b-83031dcdb7df admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4a54d9238414d4aa2c6777b0dbca15e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:04.938477 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1c5df-8cdb-4df9-b76b-83031dcdb7df admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:04.971772 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:04.986253 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-44a1c5df-8cdb-4df9-b76b-83031dcdb7df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4a54d9238414d4aa2c6777b0dbca15e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:04.999011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-44a1c5df-8cdb-4df9-b76b-83031dcdb7df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4a54d9238414d4aa2c6777b0dbca15e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:05.006362 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-44a1c5df-8cdb-4df9-b76b-83031dcdb7df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4a54d9238414d4aa2c6777b0dbca15e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:05.006975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-44a1c5df-8cdb-4df9-b76b-83031dcdb7df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4a54d9238414d4aa2c6777b0dbca15e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:05.013760 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-44a1c5df-8cdb-4df9-b76b-83031dcdb7df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4a54d9238414d4aa2c6777b0dbca15e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:05.019698 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1211/2419] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:04 2022] DELETE /identity/v3/users/a4a54d9238414d4aa2c6777b0dbca15e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:05.024157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc454bf0-f013-4bc8-aee2-c4baf755ed8e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:05.024449 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc454bf0-f013-4bc8-aee2-c4baf755ed8e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:05.024732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc454bf0-f013-4bc8-aee2-c4baf755ed8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:05.043399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-dc454bf0-f013-4bc8-aee2-c4baf755ed8e None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:05.044623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dc454bf0-f013-4bc8-aee2-c4baf755ed8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:05.105542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1209/2420] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:05.115911 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1c5df-8cdb-4df9-b76b-83031dcdb7df admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:05.117386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-44a1c5df-8cdb-4df9-b76b-83031dcdb7df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:05.174475 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1c5df-8cdb-4df9-b76b-83031dcdb7df admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:05.178179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f42c7c-4f13-423d-be1f-1c92fd31f3ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:05.178771 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f42c7c-4f13-423d-be1f-1c92fd31f3ae service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:05.179049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f42c7c-4f13-423d-be1f-1c92fd31f3ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:05.179321 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f42c7c-4f13-423d-be1f-1c92fd31f3ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:05.188431 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f42c7c-4f13-423d-be1f-1c92fd31f3ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:05.189831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f42c7c-4f13-423d-be1f-1c92fd31f3ae service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:05.199042 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1212/2421] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:05.464284 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc454bf0-f013-4bc8-aee2-c4baf755ed8e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:05.465706 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dc454bf0-f013-4bc8-aee2-c4baf755ed8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:05.477995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc454bf0-f013-4bc8-aee2-c4baf755ed8e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:05.483318 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a72370-8546-4eb8-8242-c0c70b35b3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:05.483950 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a72370-8546-4eb8-8242-c0c70b35b3d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:05.484252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a72370-8546-4eb8-8242-c0c70b35b3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:05.484519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a72370-8546-4eb8-8242-c0c70b35b3d4 admin admin] PATH_INFO: `/v3/projects/fd9519de9f474cfabafdb669b7e76269` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:05.485639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a72370-8546-4eb8-8242-c0c70b35b3d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd9519de9f474cfabafdb669b7e76269)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:05.487546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a72370-8546-4eb8-8242-c0c70b35b3d4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:05.490221 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:05.534662 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-41a72370-8546-4eb8-8242-c0c70b35b3d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd9519de9f474cfabafdb669b7e76269', 'request_id': 'req-41a72370-8546-4eb8-8242-c0c70b35b3d4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:05.540711 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-41a72370-8546-4eb8-8242-c0c70b35b3d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd9519de9f474cfabafdb669b7e76269', 'request_id': 'req-41a72370-8546-4eb8-8242-c0c70b35b3d4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:05.547886 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-41a72370-8546-4eb8-8242-c0c70b35b3d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd9519de9f474cfabafdb669b7e76269', 'request_id': 'req-41a72370-8546-4eb8-8242-c0c70b35b3d4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:05.560471 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-41a72370-8546-4eb8-8242-c0c70b35b3d4 admin admin] The token cache is being invalidate because project fd9519de9f474cfabafdb669b7e76269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:05.560768 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-41a72370-8546-4eb8-8242-c0c70b35b3d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:05.561377 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-41a72370-8546-4eb8-8242-c0c70b35b3d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:05.563194 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1210/2422] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:05 2022] DELETE /identity/v3/projects/fd9519de9f474cfabafdb669b7e76269 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:05.572353 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f42c7c-4f13-423d-be1f-1c92fd31f3ae service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:05.581919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f42c7c-4f13-423d-be1f-1c92fd31f3ae service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:05.641706 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde0188-0189-434d-a4ae-458adceb6af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:05.642347 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde0188-0189-434d-a4ae-458adceb6af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:05.642625 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde0188-0189-434d-a4ae-458adceb6af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:05.642898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde0188-0189-434d-a4ae-458adceb6af8 admin admin] PATH_INFO: `/v3/users/8b83e3c8c834413d85c0c5e868947fd9` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:05.644033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde0188-0189-434d-a4ae-458adceb6af8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b83e3c8c834413d85c0c5e868947fd9)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:05.645958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde0188-0189-434d-a4ae-458adceb6af8 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:05.696787 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-adde0188-0189-434d-a4ae-458adceb6af8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b83e3c8c834413d85c0c5e868947fd9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:05.702875 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:05.712626 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-adde0188-0189-434d-a4ae-458adceb6af8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b83e3c8c834413d85c0c5e868947fd9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:05.720305 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-adde0188-0189-434d-a4ae-458adceb6af8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b83e3c8c834413d85c0c5e868947fd9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:05.720926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-adde0188-0189-434d-a4ae-458adceb6af8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b83e3c8c834413d85c0c5e868947fd9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:05.728068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-adde0188-0189-434d-a4ae-458adceb6af8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b83e3c8c834413d85c0c5e868947fd9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:05.735361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1213/2423] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:05 2022] DELETE /identity/v3/users/8b83e3c8c834413d85c0c5e868947fd9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:06.014556 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a72370-8546-4eb8-8242-c0c70b35b3d4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:06.015989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-41a72370-8546-4eb8-8242-c0c70b35b3d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:06.029194 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a72370-8546-4eb8-8242-c0c70b35b3d4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:06.087688 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3fca6-c020-421f-9c0f-3c4e8bdfd8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:06.088345 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3fca6-c020-421f-9c0f-3c4e8bdfd8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:06.088623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3fca6-c020-421f-9c0f-3c4e8bdfd8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:06.088891 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3fca6-c020-421f-9c0f-3c4e8bdfd8f0 admin admin] PATH_INFO: `/v3/projects/c4cfce19806d4f7ea1ba06e8c21788a0` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:06.090012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3fca6-c020-421f-9c0f-3c4e8bdfd8f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4cfce19806d4f7ea1ba06e8c21788a0)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:06.092551 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3fca6-c020-421f-9c0f-3c4e8bdfd8f0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:06.139567 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-52e3fca6-c020-421f-9c0f-3c4e8bdfd8f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4cfce19806d4f7ea1ba06e8c21788a0', 'request_id': 'req-52e3fca6-c020-421f-9c0f-3c4e8bdfd8f0'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:06.145227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-52e3fca6-c020-421f-9c0f-3c4e8bdfd8f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4cfce19806d4f7ea1ba06e8c21788a0', 'request_id': 'req-52e3fca6-c020-421f-9c0f-3c4e8bdfd8f0'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:06.151355 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-52e3fca6-c020-421f-9c0f-3c4e8bdfd8f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4cfce19806d4f7ea1ba06e8c21788a0', 'request_id': 'req-52e3fca6-c020-421f-9c0f-3c4e8bdfd8f0'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:06.165235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-52e3fca6-c020-421f-9c0f-3c4e8bdfd8f0 admin admin] The token cache is being invalidate because project c4cfce19806d4f7ea1ba06e8c21788a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:06.165524 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-52e3fca6-c020-421f-9c0f-3c4e8bdfd8f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:06.166143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-52e3fca6-c020-421f-9c0f-3c4e8bdfd8f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:06.167702 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1211/2424] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:06 2022] DELETE /identity/v3/projects/c4cfce19806d4f7ea1ba06e8c21788a0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:06.173203 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026238a1-2354-45d7-9847-a53abcfc18d6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:06.173689 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026238a1-2354-45d7-9847-a53abcfc18d6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:06.173796 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026238a1-2354-45d7-9847-a53abcfc18d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:06.193202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-026238a1-2354-45d7-9847-a53abcfc18d6 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:06.194388 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-026238a1-2354-45d7-9847-a53abcfc18d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:06.252744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1214/2425] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:06.265755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7d8e9-7f76-4a8b-9331-0b59991d406b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:06.266036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7d8e9-7f76-4a8b-9331-0b59991d406b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:06.266310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7d8e9-7f76-4a8b-9331-0b59991d406b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:06.287894 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-eeb7d8e9-7f76-4a8b-9331-0b59991d406b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:06.289115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-eeb7d8e9-7f76-4a8b-9331-0b59991d406b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:06.297579 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1212/2426] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:06.301831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026238a1-2354-45d7-9847-a53abcfc18d6 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:06.311903 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026238a1-2354-45d7-9847-a53abcfc18d6 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:06.316893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686b8fd-41c1-4d67-9d36-ce4ca1221172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:06.317485 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686b8fd-41c1-4d67-9d36-ce4ca1221172 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:06.317762 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686b8fd-41c1-4d67-9d36-ce4ca1221172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:06.318033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686b8fd-41c1-4d67-9d36-ce4ca1221172 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:06.318671 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686b8fd-41c1-4d67-9d36-ce4ca1221172 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:02:06.318969 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686b8fd-41c1-4d67-9d36-ce4ca1221172 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:06.320578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686b8fd-41c1-4d67-9d36-ce4ca1221172 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:06.331887 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1215/2427] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 08:02:06.335927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7d8e9-7f76-4a8b-9331-0b59991d406b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:06.345722 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7d8e9-7f76-4a8b-9331-0b59991d406b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:06.350256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf717db3-777b-4816-bba7-c18d8bfbc1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:06.350838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf717db3-777b-4816-bba7-c18d8bfbc1a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:06.351111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf717db3-777b-4816-bba7-c18d8bfbc1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:06.351379 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf717db3-777b-4816-bba7-c18d8bfbc1a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:06.352002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf717db3-777b-4816-bba7-c18d8bfbc1a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:06.353921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf717db3-777b-4816-bba7-c18d8bfbc1a8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:06.369033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:06.386295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1213/2428] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:06 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:06.387432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686b8fd-41c1-4d67-9d36-ce4ca1221172 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:06.398142 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686b8fd-41c1-4d67-9d36-ce4ca1221172 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:06.457585 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec89d7-e625-4367-90a4-54b5450f390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:06.458220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec89d7-e625-4367-90a4-54b5450f390b admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:06.458641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec89d7-e625-4367-90a4-54b5450f390b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:06.458925 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec89d7-e625-4367-90a4-54b5450f390b admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:06.459565 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec89d7-e625-4367-90a4-54b5450f390b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:06.461514 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec89d7-e625-4367-90a4-54b5450f390b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:06.483666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1216/2429] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:06 2022] POST /identity/v3/users => generated 282 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:06.487894 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf717db3-777b-4816-bba7-c18d8bfbc1a8 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:06.498728 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf717db3-777b-4816-bba7-c18d8bfbc1a8 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:06.503523 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959dca9-6fdb-4216-9227-f356024e1c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:06.504153 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959dca9-6fdb-4216-9227-f356024e1c71 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:06.504433 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959dca9-6fdb-4216-9227-f356024e1c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:06.504703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959dca9-6fdb-4216-9227-f356024e1c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:06.505393 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959dca9-6fdb-4216-9227-f356024e1c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:06.507028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959dca9-6fdb-4216-9227-f356024e1c71 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:06.517545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1214/2430] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:06.521544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec89d7-e625-4367-90a4-54b5450f390b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:06.532050 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec89d7-e625-4367-90a4-54b5450f390b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:06.536659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59518fe-e428-4451-a479-44bb8524945b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:06.537243 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59518fe-e428-4451-a479-44bb8524945b admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:06.537569 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59518fe-e428-4451-a479-44bb8524945b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:06.538095 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59518fe-e428-4451-a479-44bb8524945b admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:06.538892 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59518fe-e428-4451-a479-44bb8524945b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:06.540509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59518fe-e428-4451-a479-44bb8524945b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:06.551148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1217/2431] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:06.555218 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959dca9-6fdb-4216-9227-f356024e1c71 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:06.565562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959dca9-6fdb-4216-9227-f356024e1c71 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:06.570182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfa579-f641-467a-970f-15990a6d94af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:06.570822 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfa579-f641-467a-970f-15990a6d94af admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:06.571102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfa579-f641-467a-970f-15990a6d94af admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:06.571376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfa579-f641-467a-970f-15990a6d94af admin admin] PATH_INFO: `/v3/projects/99fde04f1b584590b5e2437de5137592/users/8b281b3976534d69a13b3e76051c4fd6/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:06.587262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfa579-f641-467a-970f-15990a6d94af admin admin] RBAC: Authorizing `identity:create_grant(project_id=99fde04f1b584590b5e2437de5137592, user_id=8b281b3976534d69a13b3e76051c4fd6, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:06.590080 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfa579-f641-467a-970f-15990a6d94af admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:06.603400 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1215/2432] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:06 2022] PUT /identity/v3/projects/99fde04f1b584590b5e2437de5137592/users/8b281b3976534d69a13b3e76051c4fd6/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:06.614736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c227ca-f8a6-4f34-be15-52831ce6f7ee None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:06.615024 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c227ca-f8a6-4f34-be15-52831ce6f7ee None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:06.615300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c227ca-f8a6-4f34-be15-52831ce6f7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:06.643212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-03c227ca-f8a6-4f34-be15-52831ce6f7ee None None] MFA Rules not processed for user `8b281b3976534d69a13b3e76051c4fd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:06.644467 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-03c227ca-f8a6-4f34-be15-52831ce6f7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:06.671463 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:06.710121 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1218/2433] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:06.714630 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfa579-f641-467a-970f-15990a6d94af admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:06.778706 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfa579-f641-467a-970f-15990a6d94af admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:06.783779 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a408ee9-eec7-49a4-a0f4-201bec167748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:06.784409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a408ee9-eec7-49a4-a0f4-201bec167748 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:06.784688 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a408ee9-eec7-49a4-a0f4-201bec167748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:06.784960 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a408ee9-eec7-49a4-a0f4-201bec167748 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:06.785591 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a408ee9-eec7-49a4-a0f4-201bec167748 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:06.787485 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a408ee9-eec7-49a4-a0f4-201bec167748 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:06.816567 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1216/2434] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:06 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:06.820842 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c227ca-f8a6-4f34-be15-52831ce6f7ee None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:06.831919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c227ca-f8a6-4f34-be15-52831ce6f7ee None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:06.886021 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37be1d-b034-4667-b852-68db749a6cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:06.886665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37be1d-b034-4667-b852-68db749a6cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:06.886939 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37be1d-b034-4667-b852-68db749a6cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:06.887209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37be1d-b034-4667-b852-68db749a6cc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:06.887854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37be1d-b034-4667-b852-68db749a6cc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:06.890742 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37be1d-b034-4667-b852-68db749a6cc6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:06.912576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1219/2435] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:06 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:06.916619 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a408ee9-eec7-49a4-a0f4-201bec167748 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:06.926802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a408ee9-eec7-49a4-a0f4-201bec167748 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:06.931326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed6aba-2873-49f8-b1e0-1776f8b9cd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:06.931914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed6aba-2873-49f8-b1e0-1776f8b9cd16 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:06.932225 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed6aba-2873-49f8-b1e0-1776f8b9cd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:06.932495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed6aba-2873-49f8-b1e0-1776f8b9cd16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:06.933118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed6aba-2873-49f8-b1e0-1776f8b9cd16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:06.934700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed6aba-2873-49f8-b1e0-1776f8b9cd16 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:06.946762 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1217/2436] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:06.950785 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37be1d-b034-4667-b852-68db749a6cc6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:06.961534 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37be1d-b034-4667-b852-68db749a6cc6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:06.966164 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a5d6e-a789-4e9b-a381-74a343c09368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:06.966838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a5d6e-a789-4e9b-a381-74a343c09368 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:06.967080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a5d6e-a789-4e9b-a381-74a343c09368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:06.967358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a5d6e-a789-4e9b-a381-74a343c09368 admin admin] PATH_INFO: `/v3/projects/94bc00d467c6425bb8023d2120b70d76/users/c509968e894e45d8878f2e368e0bc016/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:06.982107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a5d6e-a789-4e9b-a381-74a343c09368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94bc00d467c6425bb8023d2120b70d76, user_id=c509968e894e45d8878f2e368e0bc016, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:06.984853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a5d6e-a789-4e9b-a381-74a343c09368 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:06.997553 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1220/2437] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:06 2022] PUT /identity/v3/projects/94bc00d467c6425bb8023d2120b70d76/users/c509968e894e45d8878f2e368e0bc016/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:07.002169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed6aba-2873-49f8-b1e0-1776f8b9cd16 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:07.011930 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed6aba-2873-49f8-b1e0-1776f8b9cd16 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:07.077353 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea807bf-10b9-4499-8ca8-ee723eba1a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:07.077956 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea807bf-10b9-4499-8ca8-ee723eba1a0a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:07.078234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea807bf-10b9-4499-8ca8-ee723eba1a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:07.078502 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea807bf-10b9-4499-8ca8-ee723eba1a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:07.079125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea807bf-10b9-4499-8ca8-ee723eba1a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:07.080768 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea807bf-10b9-4499-8ca8-ee723eba1a0a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:07.099050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1218/2438] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:07.103391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a5d6e-a789-4e9b-a381-74a343c09368 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:07.115016 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a5d6e-a789-4e9b-a381-74a343c09368 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:07.120709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d515de0-6214-4ec9-8b92-eb3240e98951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:07.121344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d515de0-6214-4ec9-8b92-eb3240e98951 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:07.121636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d515de0-6214-4ec9-8b92-eb3240e98951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:07.121912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d515de0-6214-4ec9-8b92-eb3240e98951 admin admin] PATH_INFO: `/v3/projects/94bc00d467c6425bb8023d2120b70d76/users/c509968e894e45d8878f2e368e0bc016/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:07.124752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d515de0-6214-4ec9-8b92-eb3240e98951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94bc00d467c6425bb8023d2120b70d76, user_id=c509968e894e45d8878f2e368e0bc016, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:07.127889 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d515de0-6214-4ec9-8b92-eb3240e98951 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:07.137943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1221/2439] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:07 2022] PUT /identity/v3/projects/94bc00d467c6425bb8023d2120b70d76/users/c509968e894e45d8878f2e368e0bc016/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:07.147938 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9dca5-79d5-4cf3-849d-ff7a0d1b08f4 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:07.148237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9dca5-79d5-4cf3-849d-ff7a0d1b08f4 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:07.148540 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9dca5-79d5-4cf3-849d-ff7a0d1b08f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:07.176389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c4e9dca5-79d5-4cf3-849d-ff7a0d1b08f4 None None] MFA Rules not processed for user `c509968e894e45d8878f2e368e0bc016`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:07.177595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c4e9dca5-79d5-4cf3-849d-ff7a0d1b08f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:07.195057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:07.246334 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1219/2440] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:07.258499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d515de0-6214-4ec9-8b92-eb3240e98951 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:07.260324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4d515de0-6214-4ec9-8b92-eb3240e98951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:07.317009 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d515de0-6214-4ec9-8b92-eb3240e98951 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:07.320686 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e1fd4-9e84-4785-b465-baac60a51718 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:07.321300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e1fd4-9e84-4785-b465-baac60a51718 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:07.321579 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e1fd4-9e84-4785-b465-baac60a51718 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:07.321857 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e1fd4-9e84-4785-b465-baac60a51718 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:07.331222 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e1fd4-9e84-4785-b465-baac60a51718 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:07.332687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e1fd4-9e84-4785-b465-baac60a51718 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:07.342052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1222/2441] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:07.750247 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9dca5-79d5-4cf3-849d-ff7a0d1b08f4 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:07.812196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9dca5-79d5-4cf3-849d-ff7a0d1b08f4 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:07.816972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde01046-63e6-45c4-9692-40648732df03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:07.817598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde01046-63e6-45c4-9692-40648732df03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:07.817882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde01046-63e6-45c4-9692-40648732df03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:07.818154 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde01046-63e6-45c4-9692-40648732df03 admin admin] PATH_INFO: `/v3/users/8b281b3976534d69a13b3e76051c4fd6` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:07.819604 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde01046-63e6-45c4-9692-40648732df03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b281b3976534d69a13b3e76051c4fd6)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:07.821593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde01046-63e6-45c4-9692-40648732df03 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:07.869511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-dde01046-63e6-45c4-9692-40648732df03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b281b3976534d69a13b3e76051c4fd6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:07.870172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-dde01046-63e6-45c4-9692-40648732df03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b281b3976534d69a13b3e76051c4fd6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:07.882096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-dde01046-63e6-45c4-9692-40648732df03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b281b3976534d69a13b3e76051c4fd6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:07.882727 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-dde01046-63e6-45c4-9692-40648732df03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b281b3976534d69a13b3e76051c4fd6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:07.890414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-dde01046-63e6-45c4-9692-40648732df03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b281b3976534d69a13b3e76051c4fd6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:07.903162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1220/2442] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:07 2022] DELETE /identity/v3/users/8b281b3976534d69a13b3e76051c4fd6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:07.908461 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066da07-646a-46c1-8962-95ff21171fda None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:07.908690 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066da07-646a-46c1-8962-95ff21171fda None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:07.908959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066da07-646a-46c1-8962-95ff21171fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:07.927780 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7066da07-646a-46c1-8962-95ff21171fda None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:07.929022 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7066da07-646a-46c1-8962-95ff21171fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:07.945708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:07.988554 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1223/2443] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:07.998113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde01046-63e6-45c4-9692-40648732df03 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:07.999489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dde01046-63e6-45c4-9692-40648732df03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:08.035387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:08.059916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde01046-63e6-45c4-9692-40648732df03 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:08.064012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af924b0b-62b9-43c1-9b44-92208f5f197b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:08.064631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af924b0b-62b9-43c1-9b44-92208f5f197b service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:08.064907 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af924b0b-62b9-43c1-9b44-92208f5f197b service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:08.065173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af924b0b-62b9-43c1-9b44-92208f5f197b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:08.074052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af924b0b-62b9-43c1-9b44-92208f5f197b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:08.075492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af924b0b-62b9-43c1-9b44-92208f5f197b service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:08.085611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1221/2444] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:02:08.357360 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066da07-646a-46c1-8962-95ff21171fda None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:08.359233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7066da07-646a-46c1-8962-95ff21171fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:08.372452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066da07-646a-46c1-8962-95ff21171fda None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:08.374784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af924b0b-62b9-43c1-9b44-92208f5f197b service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:08.376285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-af924b0b-62b9-43c1-9b44-92208f5f197b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:08.381651 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d5a39-4886-4374-b41b-db83040bcfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:08.382381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d5a39-4886-4374-b41b-db83040bcfa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:08.382748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d5a39-4886-4374-b41b-db83040bcfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:08.383100 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d5a39-4886-4374-b41b-db83040bcfa9 admin admin] PATH_INFO: `/v3/projects/99fde04f1b584590b5e2437de5137592` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:08.384778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d5a39-4886-4374-b41b-db83040bcfa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99fde04f1b584590b5e2437de5137592)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:08.386904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d5a39-4886-4374-b41b-db83040bcfa9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:08.439245 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a10d5a39-4886-4374-b41b-db83040bcfa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99fde04f1b584590b5e2437de5137592', 'request_id': 'req-a10d5a39-4886-4374-b41b-db83040bcfa9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:08.446339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a10d5a39-4886-4374-b41b-db83040bcfa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99fde04f1b584590b5e2437de5137592', 'request_id': 'req-a10d5a39-4886-4374-b41b-db83040bcfa9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:08.453132 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a10d5a39-4886-4374-b41b-db83040bcfa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99fde04f1b584590b5e2437de5137592', 'request_id': 'req-a10d5a39-4886-4374-b41b-db83040bcfa9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:08.461191 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af924b0b-62b9-43c1-9b44-92208f5f197b service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:08.466491 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a10d5a39-4886-4374-b41b-db83040bcfa9 admin admin] The token cache is being invalidate because project 99fde04f1b584590b5e2437de5137592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:08.467010 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a10d5a39-4886-4374-b41b-db83040bcfa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:08.467484 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a10d5a39-4886-4374-b41b-db83040bcfa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:08.469029 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1224/2445] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:08 2022] DELETE /identity/v3/projects/99fde04f1b584590b5e2437de5137592 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:08.472846 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d5a39-4886-4374-b41b-db83040bcfa9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:08.474157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a10d5a39-4886-4374-b41b-db83040bcfa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:08.525929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e69d5-e97b-4e9e-914c-870afdaf8c8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:08.526540 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e69d5-e97b-4e9e-914c-870afdaf8c8b service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:08.526820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e69d5-e97b-4e9e-914c-870afdaf8c8b service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:08.527092 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e69d5-e97b-4e9e-914c-870afdaf8c8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:08.528801 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ed8e69d5-e97b-4e9e-914c-870afdaf8c8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:08.538915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d5a39-4886-4374-b41b-db83040bcfa9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:08.543741 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9d51d-2449-4df0-93e6-c5b8dc88ab5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:08.544399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9d51d-2449-4df0-93e6-c5b8dc88ab5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:08.544705 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9d51d-2449-4df0-93e6-c5b8dc88ab5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:08.545011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9d51d-2449-4df0-93e6-c5b8dc88ab5f admin admin] PATH_INFO: `/v3/users/c509968e894e45d8878f2e368e0bc016` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:08.546280 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df9d51d-2449-4df0-93e6-c5b8dc88ab5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c509968e894e45d8878f2e368e0bc016)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:08.548248 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df9d51d-2449-4df0-93e6-c5b8dc88ab5f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:08.590275 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e69d5-e97b-4e9e-914c-870afdaf8c8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:08.592121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e69d5-e97b-4e9e-914c-870afdaf8c8b service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:08.595552 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8df9d51d-2449-4df0-93e6-c5b8dc88ab5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c509968e894e45d8878f2e368e0bc016'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:08.596685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:08.601306 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1222/2446] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:02:08.612092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8df9d51d-2449-4df0-93e6-c5b8dc88ab5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c509968e894e45d8878f2e368e0bc016'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:08.619410 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8df9d51d-2449-4df0-93e6-c5b8dc88ab5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c509968e894e45d8878f2e368e0bc016'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:08.620119 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8df9d51d-2449-4df0-93e6-c5b8dc88ab5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c509968e894e45d8878f2e368e0bc016'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:08.627298 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8df9d51d-2449-4df0-93e6-c5b8dc88ab5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c509968e894e45d8878f2e368e0bc016'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:08.633199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1225/2447] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:08 2022] DELETE /identity/v3/users/c509968e894e45d8878f2e368e0bc016 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:08.936972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e69d5-e97b-4e9e-914c-870afdaf8c8b service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:08.953341 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e69d5-e97b-4e9e-914c-870afdaf8c8b service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:08.971196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:09.036773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb8245-0d3f-4906-86c1-59c96064c9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:09.036773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb8245-0d3f-4906-86c1-59c96064c9d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:09.036773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb8245-0d3f-4906-86c1-59c96064c9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:09.036773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb8245-0d3f-4906-86c1-59c96064c9d6 admin admin] PATH_INFO: `/v3/projects/94bc00d467c6425bb8023d2120b70d76` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:09.037126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb8245-0d3f-4906-86c1-59c96064c9d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94bc00d467c6425bb8023d2120b70d76)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:09.037126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb8245-0d3f-4906-86c1-59c96064c9d6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:09.086333 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-37fb8245-0d3f-4906-86c1-59c96064c9d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94bc00d467c6425bb8023d2120b70d76', 'request_id': 'req-37fb8245-0d3f-4906-86c1-59c96064c9d6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:09.091700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-37fb8245-0d3f-4906-86c1-59c96064c9d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94bc00d467c6425bb8023d2120b70d76', 'request_id': 'req-37fb8245-0d3f-4906-86c1-59c96064c9d6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:09.099363 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-37fb8245-0d3f-4906-86c1-59c96064c9d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94bc00d467c6425bb8023d2120b70d76', 'request_id': 'req-37fb8245-0d3f-4906-86c1-59c96064c9d6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:09.112722 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-37fb8245-0d3f-4906-86c1-59c96064c9d6 admin admin] The token cache is being invalidate because project 94bc00d467c6425bb8023d2120b70d76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:09.113022 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-37fb8245-0d3f-4906-86c1-59c96064c9d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:09.113668 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-37fb8245-0d3f-4906-86c1-59c96064c9d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:09.115375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1223/2448] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:08 2022] DELETE /identity/v3/projects/94bc00d467c6425bb8023d2120b70d76 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:09.119895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f87bdc-3c66-4864-a626-302bfbf51c85 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:09.120208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f87bdc-3c66-4864-a626-302bfbf51c85 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:09.120481 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f87bdc-3c66-4864-a626-302bfbf51c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:09.141333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-38f87bdc-3c66-4864-a626-302bfbf51c85 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:09.142551 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-38f87bdc-3c66-4864-a626-302bfbf51c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:09.206621 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1226/2449] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:09.219021 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb56bb2-428b-4cf2-bb11-13fc4b6990c4 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:09.219307 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb56bb2-428b-4cf2-bb11-13fc4b6990c4 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:09.219577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb56bb2-428b-4cf2-bb11-13fc4b6990c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:09.241349 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9cb56bb2-428b-4cf2-bb11-13fc4b6990c4 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:09.242656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9cb56bb2-428b-4cf2-bb11-13fc4b6990c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:09.250979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1224/2450] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:09.255411 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f87bdc-3c66-4864-a626-302bfbf51c85 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:09.265461 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f87bdc-3c66-4864-a626-302bfbf51c85 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:09.270070 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35e1b1-4df4-4dd4-bcf6-12870d6486de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:09.270800 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35e1b1-4df4-4dd4-bcf6-12870d6486de admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:09.271096 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35e1b1-4df4-4dd4-bcf6-12870d6486de admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:09.271441 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35e1b1-4df4-4dd4-bcf6-12870d6486de admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:09.272055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35e1b1-4df4-4dd4-bcf6-12870d6486de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:02:09.272389 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35e1b1-4df4-4dd4-bcf6-12870d6486de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:09.273974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35e1b1-4df4-4dd4-bcf6-12870d6486de admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:09.290170 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1227/2451] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 08:02:09.291287 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb56bb2-428b-4cf2-bb11-13fc4b6990c4 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:09.301216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb56bb2-428b-4cf2-bb11-13fc4b6990c4 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:09.305732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef33d64-a0f3-403e-9f26-263286e25978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:09.306321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef33d64-a0f3-403e-9f26-263286e25978 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:09.306598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef33d64-a0f3-403e-9f26-263286e25978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:09.306865 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef33d64-a0f3-403e-9f26-263286e25978 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:09.307481 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef33d64-a0f3-403e-9f26-263286e25978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:09.309378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef33d64-a0f3-403e-9f26-263286e25978 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:09.336754 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1225/2452] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:09 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:09.340806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35e1b1-4df4-4dd4-bcf6-12870d6486de admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:09.352246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35e1b1-4df4-4dd4-bcf6-12870d6486de admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:09.412837 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aba5be-bfa8-430c-8e0a-648550bfee3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:09.413429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aba5be-bfa8-430c-8e0a-648550bfee3e admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:09.413709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aba5be-bfa8-430c-8e0a-648550bfee3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:09.413981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aba5be-bfa8-430c-8e0a-648550bfee3e admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:09.414615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aba5be-bfa8-430c-8e0a-648550bfee3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:09.416674 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aba5be-bfa8-430c-8e0a-648550bfee3e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:09.437979 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1228/2453] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:09 2022] POST /identity/v3/users => generated 282 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:09.442051 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef33d64-a0f3-403e-9f26-263286e25978 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:09.454046 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef33d64-a0f3-403e-9f26-263286e25978 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:09.461451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe22272-0539-47cd-8842-731201cc9723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:09.462807 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe22272-0539-47cd-8842-731201cc9723 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:09.463081 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe22272-0539-47cd-8842-731201cc9723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:09.463352 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe22272-0539-47cd-8842-731201cc9723 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:09.464039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe22272-0539-47cd-8842-731201cc9723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:09.465658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe22272-0539-47cd-8842-731201cc9723 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:09.476911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1226/2454] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:09.480968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aba5be-bfa8-430c-8e0a-648550bfee3e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:09.491108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aba5be-bfa8-430c-8e0a-648550bfee3e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:09.495724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680494b3-9a92-41ca-8f6c-b37d5e6650ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:09.496338 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680494b3-9a92-41ca-8f6c-b37d5e6650ee admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:09.496612 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680494b3-9a92-41ca-8f6c-b37d5e6650ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:09.496882 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680494b3-9a92-41ca-8f6c-b37d5e6650ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:09.497513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680494b3-9a92-41ca-8f6c-b37d5e6650ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:09.499374 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680494b3-9a92-41ca-8f6c-b37d5e6650ee admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:09.509802 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1229/2455] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:09.514975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe22272-0539-47cd-8842-731201cc9723 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:09.524828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe22272-0539-47cd-8842-731201cc9723 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:09.530110 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd116a-faef-4192-8237-b8cec8e44517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:09.530568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd116a-faef-4192-8237-b8cec8e44517 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:09.530839 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd116a-faef-4192-8237-b8cec8e44517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:09.531104 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd116a-faef-4192-8237-b8cec8e44517 admin admin] PATH_INFO: `/v3/projects/5ceb7d5de8ee4ffe8682db76af8bb329/users/8930c8958fba4de59a2703cc51a54069/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:09.545896 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abd116a-faef-4192-8237-b8cec8e44517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ceb7d5de8ee4ffe8682db76af8bb329, user_id=8930c8958fba4de59a2703cc51a54069, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:09.548640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abd116a-faef-4192-8237-b8cec8e44517 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:09.563011 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1227/2456] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:09 2022] PUT /identity/v3/projects/5ceb7d5de8ee4ffe8682db76af8bb329/users/8930c8958fba4de59a2703cc51a54069/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:09.574755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1cbcb-a3af-462f-923c-4abd2ff3ba2e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:09.575144 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1cbcb-a3af-462f-923c-4abd2ff3ba2e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:09.575446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1cbcb-a3af-462f-923c-4abd2ff3ba2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:09.603899 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-81b1cbcb-a3af-462f-923c-4abd2ff3ba2e None None] MFA Rules not processed for user `8930c8958fba4de59a2703cc51a54069`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:09.605121 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-81b1cbcb-a3af-462f-923c-4abd2ff3ba2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:09.624126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:09.675066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1230/2457] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:09.679655 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd116a-faef-4192-8237-b8cec8e44517 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:09.722678 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:09.748553 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd116a-faef-4192-8237-b8cec8e44517 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:09.753670 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e2fe0-308f-45d9-b968-cb6d4b7982fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:09.754280 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e2fe0-308f-45d9-b968-cb6d4b7982fd admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:09.754587 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e2fe0-308f-45d9-b968-cb6d4b7982fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:09.754859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e2fe0-308f-45d9-b968-cb6d4b7982fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:09.755479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e2fe0-308f-45d9-b968-cb6d4b7982fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:09.757427 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e2fe0-308f-45d9-b968-cb6d4b7982fd admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:09.785767 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1228/2458] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:09 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:09.789763 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1cbcb-a3af-462f-923c-4abd2ff3ba2e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:09.800169 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1cbcb-a3af-462f-923c-4abd2ff3ba2e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:09.856714 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57301c-366e-468b-9784-cca22b66ece8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:09.857352 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57301c-366e-468b-9784-cca22b66ece8 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:09.857632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57301c-366e-468b-9784-cca22b66ece8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:09.857980 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57301c-366e-468b-9784-cca22b66ece8 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:09.858627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57301c-366e-468b-9784-cca22b66ece8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:09.860548 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57301c-366e-468b-9784-cca22b66ece8 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:09.881958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1231/2459] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:09 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:09.885697 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e2fe0-308f-45d9-b968-cb6d4b7982fd admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:09.896573 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e2fe0-308f-45d9-b968-cb6d4b7982fd admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:09.901082 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196efe3-82d0-4e8c-bfde-be73418774f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:09.901668 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196efe3-82d0-4e8c-bfde-be73418774f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:09.901943 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196efe3-82d0-4e8c-bfde-be73418774f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:09.902212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196efe3-82d0-4e8c-bfde-be73418774f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:09.902834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196efe3-82d0-4e8c-bfde-be73418774f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:09.904474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196efe3-82d0-4e8c-bfde-be73418774f4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:09.914861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1229/2460] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:09.918825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57301c-366e-468b-9784-cca22b66ece8 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:09.928966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57301c-366e-468b-9784-cca22b66ece8 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:09.933663 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdce373-4727-4bfe-9da9-1b2a53be1175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:09.934308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdce373-4727-4bfe-9da9-1b2a53be1175 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:09.934588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdce373-4727-4bfe-9da9-1b2a53be1175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:09.934862 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdce373-4727-4bfe-9da9-1b2a53be1175 admin admin] PATH_INFO: `/v3/projects/c2ca6b434f2346bcb2a2c9327919a240/users/66b71149348a4836b9c56d568a156a50/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:09.950340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdce373-4727-4bfe-9da9-1b2a53be1175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ca6b434f2346bcb2a2c9327919a240, user_id=66b71149348a4836b9c56d568a156a50, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:09.955250 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdce373-4727-4bfe-9da9-1b2a53be1175 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:09.970021 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1232/2461] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:09 2022] PUT /identity/v3/projects/c2ca6b434f2346bcb2a2c9327919a240/users/66b71149348a4836b9c56d568a156a50/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:09.974109 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196efe3-82d0-4e8c-bfde-be73418774f4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:09.984109 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196efe3-82d0-4e8c-bfde-be73418774f4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:10.039849 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bff13-d96a-4b37-a785-8677d0052a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:10.040482 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bff13-d96a-4b37-a785-8677d0052a4e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:10.040758 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bff13-d96a-4b37-a785-8677d0052a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:10.041025 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bff13-d96a-4b37-a785-8677d0052a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:10.041648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2bff13-d96a-4b37-a785-8677d0052a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:10.043266 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2bff13-d96a-4b37-a785-8677d0052a4e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:10.056973 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1230/2462] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:10.058228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdce373-4727-4bfe-9da9-1b2a53be1175 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:10.069377 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdce373-4727-4bfe-9da9-1b2a53be1175 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:10.074062 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7d7da-281c-4a1b-ab90-0f652888082a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:10.074715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7d7da-281c-4a1b-ab90-0f652888082a admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:10.074997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7d7da-281c-4a1b-ab90-0f652888082a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:10.075272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7d7da-281c-4a1b-ab90-0f652888082a admin admin] PATH_INFO: `/v3/projects/c2ca6b434f2346bcb2a2c9327919a240/users/66b71149348a4836b9c56d568a156a50/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:10.077966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7d7da-281c-4a1b-ab90-0f652888082a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ca6b434f2346bcb2a2c9327919a240, user_id=66b71149348a4836b9c56d568a156a50, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:10.080636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7d7da-281c-4a1b-ab90-0f652888082a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:10.091006 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1233/2463] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:10 2022] PUT /identity/v3/projects/c2ca6b434f2346bcb2a2c9327919a240/users/66b71149348a4836b9c56d568a156a50/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:10.101327 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22380fde-7b0b-4923-979f-b98880123cad None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:10.101619 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22380fde-7b0b-4923-979f-b98880123cad None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:10.101899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22380fde-7b0b-4923-979f-b98880123cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:10.130776 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-22380fde-7b0b-4923-979f-b98880123cad None None] MFA Rules not processed for user `66b71149348a4836b9c56d568a156a50`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:10.131970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-22380fde-7b0b-4923-979f-b98880123cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:10.210469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1231/2464] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:10.217910 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7d7da-281c-4a1b-ab90-0f652888082a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:10.219662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-07d7d7da-281c-4a1b-ab90-0f652888082a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:10.278794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7d7da-281c-4a1b-ab90-0f652888082a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:10.283473 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507f4d4-2f54-456a-aec2-1f7336b02429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:10.284168 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507f4d4-2f54-456a-aec2-1f7336b02429 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:10.284448 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507f4d4-2f54-456a-aec2-1f7336b02429 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:10.284721 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507f4d4-2f54-456a-aec2-1f7336b02429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:10.296982 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507f4d4-2f54-456a-aec2-1f7336b02429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:10.298400 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507f4d4-2f54-456a-aec2-1f7336b02429 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:10.314987 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1234/2465] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:10.396529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22380fde-7b0b-4923-979f-b98880123cad None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:10.397933 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-22380fde-7b0b-4923-979f-b98880123cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:10.409882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22380fde-7b0b-4923-979f-b98880123cad None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:10.413461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc305c0-8e29-4c1b-bc4e-aef59242fb40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:10.414063 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc305c0-8e29-4c1b-bc4e-aef59242fb40 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:10.414336 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc305c0-8e29-4c1b-bc4e-aef59242fb40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:10.414605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc305c0-8e29-4c1b-bc4e-aef59242fb40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:10.448811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:10.482158 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc305c0-8e29-4c1b-bc4e-aef59242fb40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:10.484340 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc305c0-8e29-4c1b-bc4e-aef59242fb40 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:10.494062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1232/2466] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:11.716414 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507f4d4-2f54-456a-aec2-1f7336b02429 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:11.717834 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1507f4d4-2f54-456a-aec2-1f7336b02429 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:11.740450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:11.793137 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507f4d4-2f54-456a-aec2-1f7336b02429 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:11.802399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93603a2d-3975-46c5-8765-ff39ab355d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:11.802399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93603a2d-3975-46c5-8765-ff39ab355d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:11.802399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93603a2d-3975-46c5-8765-ff39ab355d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:11.802399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93603a2d-3975-46c5-8765-ff39ab355d7c admin admin] PATH_INFO: `/v3/users/3ab76e7a5671435ebf3bfe817a7e4361` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:11.802763 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93603a2d-3975-46c5-8765-ff39ab355d7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ab76e7a5671435ebf3bfe817a7e4361)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:11.802763 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93603a2d-3975-46c5-8765-ff39ab355d7c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:11.854479 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-93603a2d-3975-46c5-8765-ff39ab355d7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ab76e7a5671435ebf3bfe817a7e4361'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:11.870930 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-93603a2d-3975-46c5-8765-ff39ab355d7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ab76e7a5671435ebf3bfe817a7e4361'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:11.879181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-93603a2d-3975-46c5-8765-ff39ab355d7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ab76e7a5671435ebf3bfe817a7e4361'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:11.879878 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-93603a2d-3975-46c5-8765-ff39ab355d7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ab76e7a5671435ebf3bfe817a7e4361'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:11.887268 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-93603a2d-3975-46c5-8765-ff39ab355d7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ab76e7a5671435ebf3bfe817a7e4361'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:11.896235 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1235/2467] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:11 2022] DELETE /identity/v3/users/3ab76e7a5671435ebf3bfe817a7e4361 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:11.899282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a1056-3eb7-4227-a17f-ae36c6116d06 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:11.899751 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a1056-3eb7-4227-a17f-ae36c6116d06 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:11.899845 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a1056-3eb7-4227-a17f-ae36c6116d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:11.919722 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ba3a1056-3eb7-4227-a17f-ae36c6116d06 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:11.920919 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ba3a1056-3eb7-4227-a17f-ae36c6116d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:11.983293 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1233/2468] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:11.994089 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93603a2d-3975-46c5-8765-ff39ab355d7c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:11.996060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-93603a2d-3975-46c5-8765-ff39ab355d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:12.056733 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93603a2d-3975-46c5-8765-ff39ab355d7c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:12.060397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb17c2d-829c-4a9d-ab28-55ea510703fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:12.060998 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb17c2d-829c-4a9d-ab28-55ea510703fb service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:12.061273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb17c2d-829c-4a9d-ab28-55ea510703fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:12.061656 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb17c2d-829c-4a9d-ab28-55ea510703fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:12.075162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb17c2d-829c-4a9d-ab28-55ea510703fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:12.076638 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb17c2d-829c-4a9d-ab28-55ea510703fb service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:12.087651 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1236/2469] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:12.357792 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a1056-3eb7-4227-a17f-ae36c6116d06 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:12.359273 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ba3a1056-3eb7-4227-a17f-ae36c6116d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:12.372256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a1056-3eb7-4227-a17f-ae36c6116d06 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:12.377294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:12.378068 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:12.378353 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:12.378737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5 admin admin] PATH_INFO: `/v3/projects/c1bbaaa513b64fbe9931dcd70816d965` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:12.379913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1bbaaa513b64fbe9931dcd70816d965)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:12.381916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:12.430040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1bbaaa513b64fbe9931dcd70816d965', 'request_id': 'req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:12.435714 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1bbaaa513b64fbe9931dcd70816d965', 'request_id': 'req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:12.442154 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1bbaaa513b64fbe9931dcd70816d965', 'request_id': 'req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:12.455719 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5 admin admin] The token cache is being invalidate because project c1bbaaa513b64fbe9931dcd70816d965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:12.456016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:12.456669 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:12.458286 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1234/2470] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:12 2022] DELETE /identity/v3/projects/c1bbaaa513b64fbe9931dcd70816d965 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:12.462441 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb17c2d-829c-4a9d-ab28-55ea510703fb service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:12.472674 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb17c2d-829c-4a9d-ab28-55ea510703fb service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:12.505523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:12.529828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157f577-5f5e-4134-8310-e1a908656da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:12.531331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157f577-5f5e-4134-8310-e1a908656da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:12.531699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157f577-5f5e-4134-8310-e1a908656da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:12.532023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157f577-5f5e-4134-8310-e1a908656da7 admin admin] PATH_INFO: `/v3/users/68c30a51c9cd485faacfb5da289a7bcd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:12.533163 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157f577-5f5e-4134-8310-e1a908656da7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68c30a51c9cd485faacfb5da289a7bcd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:12.535103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157f577-5f5e-4134-8310-e1a908656da7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:12.581509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6157f577-5f5e-4134-8310-e1a908656da7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68c30a51c9cd485faacfb5da289a7bcd'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:12.593985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6157f577-5f5e-4134-8310-e1a908656da7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68c30a51c9cd485faacfb5da289a7bcd'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:12.601085 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6157f577-5f5e-4134-8310-e1a908656da7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68c30a51c9cd485faacfb5da289a7bcd'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:12.601703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6157f577-5f5e-4134-8310-e1a908656da7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68c30a51c9cd485faacfb5da289a7bcd'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:12.608619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6157f577-5f5e-4134-8310-e1a908656da7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68c30a51c9cd485faacfb5da289a7bcd'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:12.614357 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1237/2471] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:12 2022] DELETE /identity/v3/users/68c30a51c9cd485faacfb5da289a7bcd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:12.884292 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:12.885744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:12.899556 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e189500-1a45-4fb2-99bf-f4c74f01c1a5 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:12.955781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:12.964229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9634d39-465d-4c66-a31a-bc956e094905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:12.965066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9634d39-465d-4c66-a31a-bc956e094905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:12.965513 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9634d39-465d-4c66-a31a-bc956e094905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:12.965899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9634d39-465d-4c66-a31a-bc956e094905 admin admin] PATH_INFO: `/v3/projects/0b6c697204804a569f78d629f566dbcd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:12.967255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9634d39-465d-4c66-a31a-bc956e094905 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6c697204804a569f78d629f566dbcd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:12.969620 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9634d39-465d-4c66-a31a-bc956e094905 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:13.015709 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b9634d39-465d-4c66-a31a-bc956e094905 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6c697204804a569f78d629f566dbcd', 'request_id': 'req-b9634d39-465d-4c66-a31a-bc956e094905'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:13.021130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b9634d39-465d-4c66-a31a-bc956e094905 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6c697204804a569f78d629f566dbcd', 'request_id': 'req-b9634d39-465d-4c66-a31a-bc956e094905'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:13.027105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b9634d39-465d-4c66-a31a-bc956e094905 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6c697204804a569f78d629f566dbcd', 'request_id': 'req-b9634d39-465d-4c66-a31a-bc956e094905'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:13.038918 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b9634d39-465d-4c66-a31a-bc956e094905 admin admin] The token cache is being invalidate because project 0b6c697204804a569f78d629f566dbcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:13.039204 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b9634d39-465d-4c66-a31a-bc956e094905 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:13.039838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b9634d39-465d-4c66-a31a-bc956e094905 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:13.041325 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1235/2472] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:12 2022] DELETE /identity/v3/projects/0b6c697204804a569f78d629f566dbcd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:13.046265 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9457cb-3009-4dd9-9612-44f413ef091e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:13.047145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9457cb-3009-4dd9-9612-44f413ef091e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:13.047145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9457cb-3009-4dd9-9612-44f413ef091e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:13.070405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ee9457cb-3009-4dd9-9612-44f413ef091e None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:13.071615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ee9457cb-3009-4dd9-9612-44f413ef091e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:13.130886 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1238/2473] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:13.139418 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b0be9-f437-4ec9-88ae-c601b35e42e6 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:13.139698 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b0be9-f437-4ec9-88ae-c601b35e42e6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:13.140173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b0be9-f437-4ec9-88ae-c601b35e42e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:13.160608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-867b0be9-f437-4ec9-88ae-c601b35e42e6 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:13.161973 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-867b0be9-f437-4ec9-88ae-c601b35e42e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:13.172451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1236/2474] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:13.176841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9457cb-3009-4dd9-9612-44f413ef091e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:13.186610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9457cb-3009-4dd9-9612-44f413ef091e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:13.192101 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a9f64-1f00-495e-927e-8698010619ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:13.192694 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a9f64-1f00-495e-927e-8698010619ba admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:13.192967 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a9f64-1f00-495e-927e-8698010619ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:13.193234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a9f64-1f00-495e-927e-8698010619ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:13.193871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a9f64-1f00-495e-927e-8698010619ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:02:13.194171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a9f64-1f00-495e-927e-8698010619ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:13.195736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a9f64-1f00-495e-927e-8698010619ba admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:13.207849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1239/2475] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:02:13.212238 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b0be9-f437-4ec9-88ae-c601b35e42e6 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:13.223182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b0be9-f437-4ec9-88ae-c601b35e42e6 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:13.228120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a29512-28b0-40ab-b68c-ed03ff12776f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:13.228800 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a29512-28b0-40ab-b68c-ed03ff12776f admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:13.229080 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a29512-28b0-40ab-b68c-ed03ff12776f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:13.229347 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a29512-28b0-40ab-b68c-ed03ff12776f admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:13.230017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a29512-28b0-40ab-b68c-ed03ff12776f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:13.231916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a29512-28b0-40ab-b68c-ed03ff12776f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:13.239246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a9f64-1f00-495e-927e-8698010619ba admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:13.240942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-637a9f64-1f00-495e-927e-8698010619ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:13.254948 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a9f64-1f00-495e-927e-8698010619ba admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:13.259677 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533654a6-5f53-4cb6-b78e-1e101c77b402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:13.260272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533654a6-5f53-4cb6-b78e-1e101c77b402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:13.260552 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533654a6-5f53-4cb6-b78e-1e101c77b402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:13.260824 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533654a6-5f53-4cb6-b78e-1e101c77b402 admin admin] PATH_INFO: `/v3/users/8930c8958fba4de59a2703cc51a54069` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:13.261264 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1237/2476] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:13 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:13.264633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533654a6-5f53-4cb6-b78e-1e101c77b402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8930c8958fba4de59a2703cc51a54069)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:13.264633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533654a6-5f53-4cb6-b78e-1e101c77b402 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:13.265907 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a29512-28b0-40ab-b68c-ed03ff12776f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:13.306443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:13.314989 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-533654a6-5f53-4cb6-b78e-1e101c77b402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8930c8958fba4de59a2703cc51a54069'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:13.327682 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-533654a6-5f53-4cb6-b78e-1e101c77b402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8930c8958fba4de59a2703cc51a54069'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:13.330709 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a29512-28b0-40ab-b68c-ed03ff12776f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:13.335366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-533654a6-5f53-4cb6-b78e-1e101c77b402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8930c8958fba4de59a2703cc51a54069'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:13.335532 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3176bef-54e7-4bd7-ae73-fe0c84f3cb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:13.336038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-533654a6-5f53-4cb6-b78e-1e101c77b402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8930c8958fba4de59a2703cc51a54069'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:13.336193 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3176bef-54e7-4bd7-ae73-fe0c84f3cb6c admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:13.336393 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3176bef-54e7-4bd7-ae73-fe0c84f3cb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:13.336666 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3176bef-54e7-4bd7-ae73-fe0c84f3cb6c admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:13.337288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3176bef-54e7-4bd7-ae73-fe0c84f3cb6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:13.339270 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3176bef-54e7-4bd7-ae73-fe0c84f3cb6c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:13.342929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-533654a6-5f53-4cb6-b78e-1e101c77b402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8930c8958fba4de59a2703cc51a54069'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:13.348975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1240/2477] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:13 2022] DELETE /identity/v3/users/8930c8958fba4de59a2703cc51a54069 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:13.353878 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3222d-ef78-4fe8-bcab-8479d4f5b03a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:13.354155 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3222d-ef78-4fe8-bcab-8479d4f5b03a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:13.354659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3222d-ef78-4fe8-bcab-8479d4f5b03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:13.365711 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1238/2478] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:13 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:13.370214 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3176bef-54e7-4bd7-ae73-fe0c84f3cb6c admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:13.371822 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a3176bef-54e7-4bd7-ae73-fe0c84f3cb6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:13.373733 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1fd3222d-ef78-4fe8-bcab-8479d4f5b03a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:13.375497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1fd3222d-ef78-4fe8-bcab-8479d4f5b03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:13.384567 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3176bef-54e7-4bd7-ae73-fe0c84f3cb6c admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:13.431984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:13.440133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1241/2479] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:13.447683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc559e-a25b-4066-9a73-2a33344f37f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:13.448464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc559e-a25b-4066-9a73-2a33344f37f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:13.448746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc559e-a25b-4066-9a73-2a33344f37f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:13.449018 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc559e-a25b-4066-9a73-2a33344f37f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:13.449944 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc559e-a25b-4066-9a73-2a33344f37f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:13.450216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3222d-ef78-4fe8-bcab-8479d4f5b03a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:13.451573 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc559e-a25b-4066-9a73-2a33344f37f8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:13.451788 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1fd3222d-ef78-4fe8-bcab-8479d4f5b03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:13.462390 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1239/2480] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:13.466132 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc559e-a25b-4066-9a73-2a33344f37f8 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:13.485339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc559e-a25b-4066-9a73-2a33344f37f8 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:13.490461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b75f97-7f48-4e9f-8fbd-0ca087bc546e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:13.491061 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b75f97-7f48-4e9f-8fbd-0ca087bc546e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:13.491337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b75f97-7f48-4e9f-8fbd-0ca087bc546e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:13.491605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b75f97-7f48-4e9f-8fbd-0ca087bc546e admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:13.493572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b75f97-7f48-4e9f-8fbd-0ca087bc546e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:13.500658 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b75f97-7f48-4e9f-8fbd-0ca087bc546e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:13.512116 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1240/2481] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:13.516590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b75f97-7f48-4e9f-8fbd-0ca087bc546e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:13.518082 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3222d-ef78-4fe8-bcab-8479d4f5b03a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:13.522660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9fa53d-ad73-449e-90d5-2acd10fafd21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:13.524044 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fa53d-ad73-449e-90d5-2acd10fafd21 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:13.524460 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fa53d-ad73-449e-90d5-2acd10fafd21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:13.525282 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fa53d-ad73-449e-90d5-2acd10fafd21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:13.527478 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b75f97-7f48-4e9f-8fbd-0ca087bc546e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:13.533173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9a735-9250-4931-ad7a-9686f5a077e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:13.534542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9a735-9250-4931-ad7a-9686f5a077e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:13.534832 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9a735-9250-4931-ad7a-9686f5a077e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:13.535108 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9a735-9250-4931-ad7a-9686f5a077e4 admin admin] PATH_INFO: `/v3/projects/8941c70cbcc6453e9faa7f360d9f70e6/users/1d55029ef11f4870908db4eb8701f61d/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:13.536326 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9fa53d-ad73-449e-90d5-2acd10fafd21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:13.539272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9fa53d-ad73-449e-90d5-2acd10fafd21 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:13.550618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9a735-9250-4931-ad7a-9686f5a077e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8941c70cbcc6453e9faa7f360d9f70e6, user_id=1d55029ef11f4870908db4eb8701f61d, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:13.553320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9a735-9250-4931-ad7a-9686f5a077e4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:13.557718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1242/2482] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:02:13.567848 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1241/2483] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:13 2022] PUT /identity/v3/projects/8941c70cbcc6453e9faa7f360d9f70e6/users/1d55029ef11f4870908db4eb8701f61d/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:13.578331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b2468-9308-4d9e-a8f5-d35384a11043 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:13.578616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b2468-9308-4d9e-a8f5-d35384a11043 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:13.578891 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b2468-9308-4d9e-a8f5-d35384a11043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:13.607206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-695b2468-9308-4d9e-a8f5-d35384a11043 None None] MFA Rules not processed for user `1d55029ef11f4870908db4eb8701f61d`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:13.608462 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-695b2468-9308-4d9e-a8f5-d35384a11043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:13.675270 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:13.695643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1243/2484] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:13.700696 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9a735-9250-4931-ad7a-9686f5a077e4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:13.770205 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9a735-9250-4931-ad7a-9686f5a077e4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:13.775323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a85484-20df-4eab-a055-ca53b30de925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:13.775935 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a85484-20df-4eab-a055-ca53b30de925 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:13.776361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a85484-20df-4eab-a055-ca53b30de925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:13.776694 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a85484-20df-4eab-a055-ca53b30de925 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:13.777403 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a85484-20df-4eab-a055-ca53b30de925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:13.779405 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a85484-20df-4eab-a055-ca53b30de925 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:13.808856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1242/2485] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:13 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:13.813354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b2468-9308-4d9e-a8f5-d35384a11043 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:13.825113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b2468-9308-4d9e-a8f5-d35384a11043 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:13.846284 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a85484-20df-4eab-a055-ca53b30de925 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:13.848714 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-10a85484-20df-4eab-a055-ca53b30de925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:13.892272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930ccff-3482-45f7-a677-08420aaee97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:13.892926 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930ccff-3482-45f7-a677-08420aaee97c admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:13.893206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930ccff-3482-45f7-a677-08420aaee97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:13.893479 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930ccff-3482-45f7-a677-08420aaee97c admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:13.894116 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930ccff-3482-45f7-a677-08420aaee97c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:13.896050 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930ccff-3482-45f7-a677-08420aaee97c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:13.919804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a85484-20df-4eab-a055-ca53b30de925 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:13.920451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1244/2486] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:13 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:13.923780 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930ccff-3482-45f7-a677-08420aaee97c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:13.924846 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4709fd-adb4-4f8c-a5c9-701453421323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:13.925474 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4709fd-adb4-4f8c-a5c9-701453421323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:13.925747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4709fd-adb4-4f8c-a5c9-701453421323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:13.926015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4709fd-adb4-4f8c-a5c9-701453421323 admin admin] PATH_INFO: `/v3/projects/5ceb7d5de8ee4ffe8682db76af8bb329` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:13.927169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4709fd-adb4-4f8c-a5c9-701453421323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ceb7d5de8ee4ffe8682db76af8bb329)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:13.929100 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4709fd-adb4-4f8c-a5c9-701453421323 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:13.933745 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930ccff-3482-45f7-a677-08420aaee97c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:13.939283 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241e194-806d-470d-85fb-8c9201ce8a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:13.939882 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241e194-806d-470d-85fb-8c9201ce8a86 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:13.940208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241e194-806d-470d-85fb-8c9201ce8a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:13.940480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241e194-806d-470d-85fb-8c9201ce8a86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:13.941123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241e194-806d-470d-85fb-8c9201ce8a86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:13.942694 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241e194-806d-470d-85fb-8c9201ce8a86 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:13.953814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1245/2487] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:13.957961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241e194-806d-470d-85fb-8c9201ce8a86 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:13.969107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241e194-806d-470d-85fb-8c9201ce8a86 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:13.970557 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:13.983066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7c4709fd-adb4-4f8c-a5c9-701453421323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ceb7d5de8ee4ffe8682db76af8bb329', 'request_id': 'req-7c4709fd-adb4-4f8c-a5c9-701453421323'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:13.989817 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7c4709fd-adb4-4f8c-a5c9-701453421323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ceb7d5de8ee4ffe8682db76af8bb329', 'request_id': 'req-7c4709fd-adb4-4f8c-a5c9-701453421323'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:13.996112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7c4709fd-adb4-4f8c-a5c9-701453421323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ceb7d5de8ee4ffe8682db76af8bb329', 'request_id': 'req-7c4709fd-adb4-4f8c-a5c9-701453421323'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:14.010501 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7c4709fd-adb4-4f8c-a5c9-701453421323 admin admin] The token cache is being invalidate because project 5ceb7d5de8ee4ffe8682db76af8bb329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:14.011170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7c4709fd-adb4-4f8c-a5c9-701453421323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:14.012059 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-7c4709fd-adb4-4f8c-a5c9-701453421323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:14.013641 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1243/2488] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:13 2022] DELETE /identity/v3/projects/5ceb7d5de8ee4ffe8682db76af8bb329 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:14.017537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4709fd-adb4-4f8c-a5c9-701453421323 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:14.019172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7c4709fd-adb4-4f8c-a5c9-701453421323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:14.034998 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170d18d-97ae-4aa4-9517-f999f429c9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:14.035670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170d18d-97ae-4aa4-9517-f999f429c9ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:14.036100 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170d18d-97ae-4aa4-9517-f999f429c9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:14.036391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170d18d-97ae-4aa4-9517-f999f429c9ff admin admin] PATH_INFO: `/v3/projects/c2740d6325ce4d23ae644d0c690a87bd/users/c2f67607a0fb426798951b09ebd4b4af/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:14.054611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170d18d-97ae-4aa4-9517-f999f429c9ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2740d6325ce4d23ae644d0c690a87bd, user_id=c2f67607a0fb426798951b09ebd4b4af, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:14.057703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170d18d-97ae-4aa4-9517-f999f429c9ff admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:14.071880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1246/2489] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:13 2022] PUT /identity/v3/projects/c2740d6325ce4d23ae644d0c690a87bd/users/c2f67607a0fb426798951b09ebd4b4af/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:14.075681 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170d18d-97ae-4aa4-9517-f999f429c9ff admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:14.077889 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8170d18d-97ae-4aa4-9517-f999f429c9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:14.089196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4709fd-adb4-4f8c-a5c9-701453421323 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:14.134657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:14.149224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170d18d-97ae-4aa4-9517-f999f429c9ff admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:14.149950 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2dc57-9744-4e46-9e5c-8ad0956e79c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:14.150533 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2dc57-9744-4e46-9e5c-8ad0956e79c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:14.150806 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2dc57-9744-4e46-9e5c-8ad0956e79c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:14.151081 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2dc57-9744-4e46-9e5c-8ad0956e79c9 admin admin] PATH_INFO: `/v3/users/66b71149348a4836b9c56d568a156a50` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:14.152220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2dc57-9744-4e46-9e5c-8ad0956e79c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66b71149348a4836b9c56d568a156a50)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:14.154120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2dc57-9744-4e46-9e5c-8ad0956e79c9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:14.154265 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a8045-88f3-4cd7-8884-41ac57829ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:14.154866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a8045-88f3-4cd7-8884-41ac57829ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:14.155143 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a8045-88f3-4cd7-8884-41ac57829ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:14.155415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a8045-88f3-4cd7-8884-41ac57829ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:14.156103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a8045-88f3-4cd7-8884-41ac57829ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:14.157757 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a8045-88f3-4cd7-8884-41ac57829ba0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:14.168367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1247/2490] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:14.173034 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a8045-88f3-4cd7-8884-41ac57829ba0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:14.183455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a8045-88f3-4cd7-8884-41ac57829ba0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:14.189729 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cff0b9-59b6-4b18-b73e-4dc3cfee2f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:14.190391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cff0b9-59b6-4b18-b73e-4dc3cfee2f80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:14.190665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cff0b9-59b6-4b18-b73e-4dc3cfee2f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:14.190968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cff0b9-59b6-4b18-b73e-4dc3cfee2f80 admin admin] PATH_INFO: `/v3/projects/c2740d6325ce4d23ae644d0c690a87bd/users/c2f67607a0fb426798951b09ebd4b4af/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:14.192902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cff0b9-59b6-4b18-b73e-4dc3cfee2f80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2740d6325ce4d23ae644d0c690a87bd, user_id=c2f67607a0fb426798951b09ebd4b4af, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:14.195514 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cff0b9-59b6-4b18-b73e-4dc3cfee2f80 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:14.201989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-aaf2dc57-9744-4e46-9e5c-8ad0956e79c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66b71149348a4836b9c56d568a156a50'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:14.202750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-aaf2dc57-9744-4e46-9e5c-8ad0956e79c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66b71149348a4836b9c56d568a156a50'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:14.205655 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1248/2491] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:14 2022] PUT /identity/v3/projects/c2740d6325ce4d23ae644d0c690a87bd/users/c2f67607a0fb426798951b09ebd4b4af/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:14.214962 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-aaf2dc57-9744-4e46-9e5c-8ad0956e79c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66b71149348a4836b9c56d568a156a50'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:14.215676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-aaf2dc57-9744-4e46-9e5c-8ad0956e79c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66b71149348a4836b9c56d568a156a50'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:14.222092 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:14.224736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-aaf2dc57-9744-4e46-9e5c-8ad0956e79c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66b71149348a4836b9c56d568a156a50'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:14.241503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1244/2492] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:14 2022] DELETE /identity/v3/users/66b71149348a4836b9c56d568a156a50 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:14.275349 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc3f0f-c599-4bb5-a86a-496691324faa None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:14.275730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc3f0f-c599-4bb5-a86a-496691324faa None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:14.275904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc3f0f-c599-4bb5-a86a-496691324faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:14.307796 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-63cc3f0f-c599-4bb5-a86a-496691324faa None None] MFA Rules not processed for user `c2f67607a0fb426798951b09ebd4b4af`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:14.309198 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-63cc3f0f-c599-4bb5-a86a-496691324faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:14.397338 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1249/2493] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:14.407070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2dc57-9744-4e46-9e5c-8ad0956e79c9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:14.408988 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-aaf2dc57-9744-4e46-9e5c-8ad0956e79c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:14.473936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2dc57-9744-4e46-9e5c-8ad0956e79c9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:14.477720 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013ecd6-3275-4445-92b0-ee5b6e5ac033 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:14.478396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013ecd6-3275-4445-92b0-ee5b6e5ac033 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:14.478682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013ecd6-3275-4445-92b0-ee5b6e5ac033 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:14.478950 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013ecd6-3275-4445-92b0-ee5b6e5ac033 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:14.488651 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013ecd6-3275-4445-92b0-ee5b6e5ac033 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:14.490078 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013ecd6-3275-4445-92b0-ee5b6e5ac033 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:14.499507 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1245/2494] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:14.524440 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc3f0f-c599-4bb5-a86a-496691324faa None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:14.526367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-63cc3f0f-c599-4bb5-a86a-496691324faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:14.630510 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc3f0f-c599-4bb5-a86a-496691324faa None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:14.635409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd4da8-45b5-4265-8d8f-42d63ab268b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:14.636130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd4da8-45b5-4265-8d8f-42d63ab268b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:14.636417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd4da8-45b5-4265-8d8f-42d63ab268b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:14.636691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd4da8-45b5-4265-8d8f-42d63ab268b2 admin admin] PATH_INFO: `/v3/projects/c2ca6b434f2346bcb2a2c9327919a240` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:14.637831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd4da8-45b5-4265-8d8f-42d63ab268b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2ca6b434f2346bcb2a2c9327919a240)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:14.640334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd4da8-45b5-4265-8d8f-42d63ab268b2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:14.693487 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-15bd4da8-45b5-4265-8d8f-42d63ab268b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2ca6b434f2346bcb2a2c9327919a240', 'request_id': 'req-15bd4da8-45b5-4265-8d8f-42d63ab268b2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:14.698964 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-15bd4da8-45b5-4265-8d8f-42d63ab268b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2ca6b434f2346bcb2a2c9327919a240', 'request_id': 'req-15bd4da8-45b5-4265-8d8f-42d63ab268b2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:14.705330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:14.706649 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-15bd4da8-45b5-4265-8d8f-42d63ab268b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2ca6b434f2346bcb2a2c9327919a240', 'request_id': 'req-15bd4da8-45b5-4265-8d8f-42d63ab268b2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:14.724096 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-15bd4da8-45b5-4265-8d8f-42d63ab268b2 admin admin] The token cache is being invalidate because project c2ca6b434f2346bcb2a2c9327919a240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:14.724349 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-15bd4da8-45b5-4265-8d8f-42d63ab268b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:14.724979 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-15bd4da8-45b5-4265-8d8f-42d63ab268b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:14.726447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1250/2495] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:14 2022] DELETE /identity/v3/projects/c2ca6b434f2346bcb2a2c9327919a240 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:14.731146 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e9387-f109-43e8-8123-15dcbfd2e15f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:14.731427 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e9387-f109-43e8-8123-15dcbfd2e15f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:14.731699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e9387-f109-43e8-8123-15dcbfd2e15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:14.751016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9b0e9387-f109-43e8-8123-15dcbfd2e15f None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:14.752386 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9b0e9387-f109-43e8-8123-15dcbfd2e15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:14.819170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1246/2496] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:14.830212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d440b-33b6-492e-a260-62ff1d26d72b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:14.830212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d440b-33b6-492e-a260-62ff1d26d72b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:14.830212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d440b-33b6-492e-a260-62ff1d26d72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:14.851309 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0b2d440b-33b6-492e-a260-62ff1d26d72b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:14.852608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0b2d440b-33b6-492e-a260-62ff1d26d72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:14.860607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1251/2497] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:14.865328 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e9387-f109-43e8-8123-15dcbfd2e15f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:14.877305 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e9387-f109-43e8-8123-15dcbfd2e15f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:14.882489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e2a0e-75f0-430c-ae95-d52bd0a36042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:14.883101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e2a0e-75f0-430c-ae95-d52bd0a36042 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:14.883376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e2a0e-75f0-430c-ae95-d52bd0a36042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:14.883645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e2a0e-75f0-430c-ae95-d52bd0a36042 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:14.884336 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e2a0e-75f0-430c-ae95-d52bd0a36042 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:02:14.884640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e2a0e-75f0-430c-ae95-d52bd0a36042 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:14.886262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e2a0e-75f0-430c-ae95-d52bd0a36042 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:14.901151 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d440b-33b6-492e-a260-62ff1d26d72b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:14.902635 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0b2d440b-33b6-492e-a260-62ff1d26d72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:14.904128 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1247/2498] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 08:02:14.907328 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e2a0e-75f0-430c-ae95-d52bd0a36042 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:14.915542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d440b-33b6-492e-a260-62ff1d26d72b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:14.916984 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e2a0e-75f0-430c-ae95-d52bd0a36042 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:14.920247 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911485b9-b774-44fe-b94c-8b75e19f9ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:14.920870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911485b9-b774-44fe-b94c-8b75e19f9ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:14.921322 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911485b9-b774-44fe-b94c-8b75e19f9ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:14.921723 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9470c4-7657-4be2-b688-140f7d0ba27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:14.922178 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911485b9-b774-44fe-b94c-8b75e19f9ca4 admin admin] PATH_INFO: `/v3/users/1d55029ef11f4870908db4eb8701f61d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:14.922305 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9470c4-7657-4be2-b688-140f7d0ba27c admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:14.922588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9470c4-7657-4be2-b688-140f7d0ba27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:14.922866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9470c4-7657-4be2-b688-140f7d0ba27c admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:14.923129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911485b9-b774-44fe-b94c-8b75e19f9ca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d55029ef11f4870908db4eb8701f61d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:14.923479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9470c4-7657-4be2-b688-140f7d0ba27c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:14.925182 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911485b9-b774-44fe-b94c-8b75e19f9ca4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:14.925397 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9470c4-7657-4be2-b688-140f7d0ba27c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:14.954847 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1248/2499] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:14 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:14.959147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9470c4-7657-4be2-b688-140f7d0ba27c admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:14.983654 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-911485b9-b774-44fe-b94c-8b75e19f9ca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d55029ef11f4870908db4eb8701f61d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:14.995687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-911485b9-b774-44fe-b94c-8b75e19f9ca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d55029ef11f4870908db4eb8701f61d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:15.004163 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-911485b9-b774-44fe-b94c-8b75e19f9ca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d55029ef11f4870908db4eb8701f61d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:15.004829 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-911485b9-b774-44fe-b94c-8b75e19f9ca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d55029ef11f4870908db4eb8701f61d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:15.012216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-911485b9-b774-44fe-b94c-8b75e19f9ca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d55029ef11f4870908db4eb8701f61d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:15.018879 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1252/2500] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:14 2022] DELETE /identity/v3/users/1d55029ef11f4870908db4eb8701f61d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:15.021152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:15.023358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e2787-8219-4b4d-8f50-c665eb351016 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:15.023887 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e2787-8219-4b4d-8f50-c665eb351016 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:15.024127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e2787-8219-4b4d-8f50-c665eb351016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:15.025830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9470c4-7657-4be2-b688-140f7d0ba27c admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:15.043350 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0c9e2787-8219-4b4d-8f50-c665eb351016 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:15.044605 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0c9e2787-8219-4b4d-8f50-c665eb351016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:15.090699 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65292d29-1617-42a1-8132-27e48b370bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:15.091309 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65292d29-1617-42a1-8132-27e48b370bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:15.091587 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65292d29-1617-42a1-8132-27e48b370bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:15.091857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65292d29-1617-42a1-8132-27e48b370bd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:15.092510 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65292d29-1617-42a1-8132-27e48b370bd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:15.094533 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65292d29-1617-42a1-8132-27e48b370bd3 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:15.115805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1253/2501] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:15.122814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1249/2502] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:14 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:15.123923 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e2787-8219-4b4d-8f50-c665eb351016 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:15.126244 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0c9e2787-8219-4b4d-8f50-c665eb351016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:15.126373 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65292d29-1617-42a1-8132-27e48b370bd3 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:15.127846 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-65292d29-1617-42a1-8132-27e48b370bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:15.138064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e2787-8219-4b4d-8f50-c665eb351016 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:15.143741 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249844b3-1b8e-42cd-9e9d-cce4669193a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:15.144433 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249844b3-1b8e-42cd-9e9d-cce4669193a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:15.144715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249844b3-1b8e-42cd-9e9d-cce4669193a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:15.144994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249844b3-1b8e-42cd-9e9d-cce4669193a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:15.145630 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249844b3-1b8e-42cd-9e9d-cce4669193a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:15.147212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249844b3-1b8e-42cd-9e9d-cce4669193a2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:15.161828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1254/2503] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:15.165904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249844b3-1b8e-42cd-9e9d-cce4669193a2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:15.175925 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249844b3-1b8e-42cd-9e9d-cce4669193a2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:15.180577 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c8617-870d-440b-a762-be7fbb2fa2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:15.181171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c8617-870d-440b-a762-be7fbb2fa2ea admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:15.181442 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c8617-870d-440b-a762-be7fbb2fa2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:15.181714 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c8617-870d-440b-a762-be7fbb2fa2ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:15.182341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c8617-870d-440b-a762-be7fbb2fa2ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:15.183898 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c8617-870d-440b-a762-be7fbb2fa2ea admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:15.187632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65292d29-1617-42a1-8132-27e48b370bd3 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:15.191539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99de00d-1205-4996-98c5-1d5503815a05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:15.192170 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99de00d-1205-4996-98c5-1d5503815a05 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:15.192453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99de00d-1205-4996-98c5-1d5503815a05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:15.192721 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99de00d-1205-4996-98c5-1d5503815a05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:15.194830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1255/2504] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:15.198840 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c8617-870d-440b-a762-be7fbb2fa2ea admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:15.204291 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99de00d-1205-4996-98c5-1d5503815a05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:15.205731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99de00d-1205-4996-98c5-1d5503815a05 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:15.209576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c8617-870d-440b-a762-be7fbb2fa2ea admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:15.214249 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64689ed-716b-497f-95ac-02d167741916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:15.214880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64689ed-716b-497f-95ac-02d167741916 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:15.215160 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64689ed-716b-497f-95ac-02d167741916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:15.215437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64689ed-716b-497f-95ac-02d167741916 admin admin] PATH_INFO: `/v3/projects/207fcea03c5d45b3b4782b66ae52f038/users/a3eeabb2bed246c9b616360a5965cc23/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:15.215565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1250/2505] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:02:15.230421 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64689ed-716b-497f-95ac-02d167741916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=207fcea03c5d45b3b4782b66ae52f038, user_id=a3eeabb2bed246c9b616360a5965cc23, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:15.233109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64689ed-716b-497f-95ac-02d167741916 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:15.247322 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1256/2506] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:15 2022] PUT /identity/v3/projects/207fcea03c5d45b3b4782b66ae52f038/users/a3eeabb2bed246c9b616360a5965cc23/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:15.259259 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a5d29-c91d-438c-9bfb-6e2bf1dd4f4c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:15.259528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a5d29-c91d-438c-9bfb-6e2bf1dd4f4c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:15.259799 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a5d29-c91d-438c-9bfb-6e2bf1dd4f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:15.290389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-416a5d29-c91d-438c-9bfb-6e2bf1dd4f4c None None] MFA Rules not processed for user `a3eeabb2bed246c9b616360a5965cc23`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:15.291602 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-416a5d29-c91d-438c-9bfb-6e2bf1dd4f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:15.367873 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1251/2507] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:15.381333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64689ed-716b-497f-95ac-02d167741916 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:15.383379 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d64689ed-716b-497f-95ac-02d167741916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:15.394521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:15.457671 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64689ed-716b-497f-95ac-02d167741916 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:15.461741 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6f016-70cb-4112-a139-0ba1d551b01a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b0652c89c9945e2866512c9da4793ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:15.462811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6f016-70cb-4112-a139-0ba1d551b01a service glance] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:15.463387 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6f016-70cb-4112-a139-0ba1d551b01a service glance] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:15.463856 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6f016-70cb-4112-a139-0ba1d551b01a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:15.476466 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6f016-70cb-4112-a139-0ba1d551b01a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:15.477841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6f016-70cb-4112-a139-0ba1d551b01a service glance] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:15.489143 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1257/2508] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:15.504246 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a5d29-c91d-438c-9bfb-6e2bf1dd4f4c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:15.506245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-416a5d29-c91d-438c-9bfb-6e2bf1dd4f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:15.544459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:15.586313 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a5d29-c91d-438c-9bfb-6e2bf1dd4f4c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:15.591283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f9e3b-030e-44ee-8375-0651061a7507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:15.592296 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f9e3b-030e-44ee-8375-0651061a7507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:15.592591 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f9e3b-030e-44ee-8375-0651061a7507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:15.592899 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f9e3b-030e-44ee-8375-0651061a7507 admin admin] PATH_INFO: `/v3/projects/8941c70cbcc6453e9faa7f360d9f70e6` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:15.594201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f9e3b-030e-44ee-8375-0651061a7507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8941c70cbcc6453e9faa7f360d9f70e6)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:15.596253 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f9e3b-030e-44ee-8375-0651061a7507 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:15.652734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b83f9e3b-030e-44ee-8375-0651061a7507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8941c70cbcc6453e9faa7f360d9f70e6', 'request_id': 'req-b83f9e3b-030e-44ee-8375-0651061a7507'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:15.660773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b83f9e3b-030e-44ee-8375-0651061a7507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8941c70cbcc6453e9faa7f360d9f70e6', 'request_id': 'req-b83f9e3b-030e-44ee-8375-0651061a7507'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:15.672433 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b83f9e3b-030e-44ee-8375-0651061a7507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8941c70cbcc6453e9faa7f360d9f70e6', 'request_id': 'req-b83f9e3b-030e-44ee-8375-0651061a7507'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:15.688927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b83f9e3b-030e-44ee-8375-0651061a7507 admin admin] The token cache is being invalidate because project 8941c70cbcc6453e9faa7f360d9f70e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:15.689221 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b83f9e3b-030e-44ee-8375-0651061a7507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:15.689964 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b83f9e3b-030e-44ee-8375-0651061a7507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:15.691992 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1252/2509] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:15 2022] DELETE /identity/v3/projects/8941c70cbcc6453e9faa7f360d9f70e6 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:15.695768 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6f016-70cb-4112-a139-0ba1d551b01a service glance] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:15.706533 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6f016-70cb-4112-a139-0ba1d551b01a service glance] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:15.764474 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff366d-a469-413e-9a59-7a118cfecb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:15.765200 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff366d-a469-413e-9a59-7a118cfecb6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:15.765620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff366d-a469-413e-9a59-7a118cfecb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:15.765919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff366d-a469-413e-9a59-7a118cfecb6f admin admin] PATH_INFO: `/v3/users/c2f67607a0fb426798951b09ebd4b4af` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:15.767043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff366d-a469-413e-9a59-7a118cfecb6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2f67607a0fb426798951b09ebd4b4af)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:15.769143 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff366d-a469-413e-9a59-7a118cfecb6f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:15.838845 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-83ff366d-a469-413e-9a59-7a118cfecb6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2f67607a0fb426798951b09ebd4b4af'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:15.859576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-83ff366d-a469-413e-9a59-7a118cfecb6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2f67607a0fb426798951b09ebd4b4af'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:15.872584 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-83ff366d-a469-413e-9a59-7a118cfecb6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2f67607a0fb426798951b09ebd4b4af'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:15.873355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-83ff366d-a469-413e-9a59-7a118cfecb6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2f67607a0fb426798951b09ebd4b4af'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:15.885191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-83ff366d-a469-413e-9a59-7a118cfecb6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2f67607a0fb426798951b09ebd4b4af'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:15.895354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1258/2510] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:15 2022] DELETE /identity/v3/users/c2f67607a0fb426798951b09ebd4b4af => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:16.209011 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f9e3b-030e-44ee-8375-0651061a7507 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:16.210784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b83f9e3b-030e-44ee-8375-0651061a7507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:16.231079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f9e3b-030e-44ee-8375-0651061a7507 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:16.289754 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90701119-684b-47a3-aafe-ea66bbc8ecf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:16.290394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90701119-684b-47a3-aafe-ea66bbc8ecf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:16.290665 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90701119-684b-47a3-aafe-ea66bbc8ecf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:16.290932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90701119-684b-47a3-aafe-ea66bbc8ecf7 admin admin] PATH_INFO: `/v3/projects/c2740d6325ce4d23ae644d0c690a87bd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:16.292027 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90701119-684b-47a3-aafe-ea66bbc8ecf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2740d6325ce4d23ae644d0c690a87bd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:16.293942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90701119-684b-47a3-aafe-ea66bbc8ecf7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:16.340068 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-90701119-684b-47a3-aafe-ea66bbc8ecf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2740d6325ce4d23ae644d0c690a87bd', 'request_id': 'req-90701119-684b-47a3-aafe-ea66bbc8ecf7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:16.345756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-90701119-684b-47a3-aafe-ea66bbc8ecf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2740d6325ce4d23ae644d0c690a87bd', 'request_id': 'req-90701119-684b-47a3-aafe-ea66bbc8ecf7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:16.346818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:16.353872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-90701119-684b-47a3-aafe-ea66bbc8ecf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2740d6325ce4d23ae644d0c690a87bd', 'request_id': 'req-90701119-684b-47a3-aafe-ea66bbc8ecf7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:16.368005 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-90701119-684b-47a3-aafe-ea66bbc8ecf7 admin admin] The token cache is being invalidate because project c2740d6325ce4d23ae644d0c690a87bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:16.368347 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-90701119-684b-47a3-aafe-ea66bbc8ecf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:16.368961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-90701119-684b-47a3-aafe-ea66bbc8ecf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:16.370454 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1253/2511] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:16 2022] DELETE /identity/v3/projects/c2740d6325ce4d23ae644d0c690a87bd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:16.376001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f45676-316c-4356-b406-fe20516d488e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:16.376638 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f45676-316c-4356-b406-fe20516d488e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:16.376638 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f45676-316c-4356-b406-fe20516d488e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:16.397749 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b4f45676-316c-4356-b406-fe20516d488e None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:16.398962 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b4f45676-316c-4356-b406-fe20516d488e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:16.435406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:16.472725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1259/2512] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:16.480672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafc18c-ca87-4098-ab6b-10f563194629 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:16.480961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafc18c-ca87-4098-ab6b-10f563194629 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:16.481243 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafc18c-ca87-4098-ab6b-10f563194629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:16.502514 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-2aafc18c-ca87-4098-ab6b-10f563194629 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:16.503752 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2aafc18c-ca87-4098-ab6b-10f563194629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:16.514182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1254/2513] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:16.518931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f45676-316c-4356-b406-fe20516d488e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:16.532616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f45676-316c-4356-b406-fe20516d488e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:16.538702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec12aa-683c-4496-8000-0693150ff6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:16.539467 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec12aa-683c-4496-8000-0693150ff6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:16.539742 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec12aa-683c-4496-8000-0693150ff6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:16.540252 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec12aa-683c-4496-8000-0693150ff6b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:16.541439 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec12aa-683c-4496-8000-0693150ff6b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:02:16.542040 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec12aa-683c-4496-8000-0693150ff6b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:16.544742 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec12aa-683c-4496-8000-0693150ff6b7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:16.562041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1260/2514] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:02:16.563671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aafc18c-ca87-4098-ab6b-10f563194629 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:16.573550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aafc18c-ca87-4098-ab6b-10f563194629 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:16.578613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af5b76-695d-417c-87f7-6c6c62f98af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:16.579214 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af5b76-695d-417c-87f7-6c6c62f98af2 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:16.579490 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af5b76-695d-417c-87f7-6c6c62f98af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:16.579794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af5b76-695d-417c-87f7-6c6c62f98af2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:16.580440 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af5b76-695d-417c-87f7-6c6c62f98af2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:16.582343 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af5b76-695d-417c-87f7-6c6c62f98af2 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:16.613814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1255/2515] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:16 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:16.619208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec12aa-683c-4496-8000-0693150ff6b7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:16.630659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec12aa-683c-4496-8000-0693150ff6b7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:16.699287 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c438aa9-6bcc-4d6a-947b-a567a80f2ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:16.699899 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c438aa9-6bcc-4d6a-947b-a567a80f2ffb admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:16.700665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c438aa9-6bcc-4d6a-947b-a567a80f2ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:16.700665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c438aa9-6bcc-4d6a-947b-a567a80f2ffb admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:16.701108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c438aa9-6bcc-4d6a-947b-a567a80f2ffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:16.703428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c438aa9-6bcc-4d6a-947b-a567a80f2ffb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:16.729333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1261/2516] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:16 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:16.735051 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af5b76-695d-417c-87f7-6c6c62f98af2 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:16.765547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af5b76-695d-417c-87f7-6c6c62f98af2 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:16.772369 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d60c2-9020-48c3-b416-b28ab37d368a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:16.772994 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d60c2-9020-48c3-b416-b28ab37d368a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:16.773263 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d60c2-9020-48c3-b416-b28ab37d368a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:16.773532 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d60c2-9020-48c3-b416-b28ab37d368a admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:16.774166 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d60c2-9020-48c3-b416-b28ab37d368a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:16.775963 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d60c2-9020-48c3-b416-b28ab37d368a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:16.789568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1256/2517] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:16.793886 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c438aa9-6bcc-4d6a-947b-a567a80f2ffb admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:16.812369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c438aa9-6bcc-4d6a-947b-a567a80f2ffb admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:16.818279 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831b392-7a06-42df-8ca2-e6dc3862bacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:16.818911 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831b392-7a06-42df-8ca2-e6dc3862bacd admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:16.819195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831b392-7a06-42df-8ca2-e6dc3862bacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:16.819469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831b392-7a06-42df-8ca2-e6dc3862bacd admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:16.821117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831b392-7a06-42df-8ca2-e6dc3862bacd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:16.822709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831b392-7a06-42df-8ca2-e6dc3862bacd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:16.839841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1262/2518] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:16.845211 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d60c2-9020-48c3-b416-b28ab37d368a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:16.856108 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d60c2-9020-48c3-b416-b28ab37d368a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:16.862536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edec8b-3e5d-45cc-b722-1e352f2aca30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:16.863198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edec8b-3e5d-45cc-b722-1e352f2aca30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:16.863478 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edec8b-3e5d-45cc-b722-1e352f2aca30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:16.863751 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edec8b-3e5d-45cc-b722-1e352f2aca30 admin admin] PATH_INFO: `/v3/projects/53f46cbeb6ad4dd58d0df45d9487c88a/users/4d4686c1d2984c6b9f738f9e1e75cdcb/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:16.880639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edec8b-3e5d-45cc-b722-1e352f2aca30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53f46cbeb6ad4dd58d0df45d9487c88a, user_id=4d4686c1d2984c6b9f738f9e1e75cdcb, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:16.883324 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edec8b-3e5d-45cc-b722-1e352f2aca30 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:16.897097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1257/2519] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:16 2022] PUT /identity/v3/projects/53f46cbeb6ad4dd58d0df45d9487c88a/users/4d4686c1d2984c6b9f738f9e1e75cdcb/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:16.908511 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998f3de-76af-4da6-9c09-dae4108726f5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:16.908742 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998f3de-76af-4da6-9c09-dae4108726f5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:16.909019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998f3de-76af-4da6-9c09-dae4108726f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:16.938738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a998f3de-76af-4da6-9c09-dae4108726f5 None None] MFA Rules not processed for user `4d4686c1d2984c6b9f738f9e1e75cdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:16.939989 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a998f3de-76af-4da6-9c09-dae4108726f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:17.011616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1263/2520] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:17.019148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edec8b-3e5d-45cc-b722-1e352f2aca30 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:17.098709 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edec8b-3e5d-45cc-b722-1e352f2aca30 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:17.104428 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562c077-e530-48c9-805d-ef7b5be8a7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:17.105305 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562c077-e530-48c9-805d-ef7b5be8a7cf admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:17.105811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562c077-e530-48c9-805d-ef7b5be8a7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:17.106292 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562c077-e530-48c9-805d-ef7b5be8a7cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:17.107028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562c077-e530-48c9-805d-ef7b5be8a7cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:17.109745 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562c077-e530-48c9-805d-ef7b5be8a7cf admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:17.143636 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1258/2521] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:17 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:17.149418 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998f3de-76af-4da6-9c09-dae4108726f5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:17.163404 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998f3de-76af-4da6-9c09-dae4108726f5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:17.199136 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:17.246280 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018563a-b4de-4fcd-a009-249bce43e14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:17.247210 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018563a-b4de-4fcd-a009-249bce43e14a admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:17.247491 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018563a-b4de-4fcd-a009-249bce43e14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:17.247761 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018563a-b4de-4fcd-a009-249bce43e14a admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:17.248451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018563a-b4de-4fcd-a009-249bce43e14a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:17.251325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018563a-b4de-4fcd-a009-249bce43e14a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:17.280613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1264/2522] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:17 2022] POST /identity/v3/users => generated 298 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:17.282664 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562c077-e530-48c9-805d-ef7b5be8a7cf admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:17.294927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562c077-e530-48c9-805d-ef7b5be8a7cf admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:17.302545 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74dc04b-df5c-45a0-ae88-a8792478a539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:17.303494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74dc04b-df5c-45a0-ae88-a8792478a539 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:17.303952 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74dc04b-df5c-45a0-ae88-a8792478a539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:17.304428 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74dc04b-df5c-45a0-ae88-a8792478a539 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:17.305208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74dc04b-df5c-45a0-ae88-a8792478a539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:17.306936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74dc04b-df5c-45a0-ae88-a8792478a539 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:17.319282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1259/2523] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:17.323851 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018563a-b4de-4fcd-a009-249bce43e14a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:17.335830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018563a-b4de-4fcd-a009-249bce43e14a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:17.339806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc78ed-687d-43c7-a63f-ad3c20f31c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:17.340474 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc78ed-687d-43c7-a63f-ad3c20f31c85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:17.340750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc78ed-687d-43c7-a63f-ad3c20f31c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:17.341019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc78ed-687d-43c7-a63f-ad3c20f31c85 admin admin] PATH_INFO: `/v3/projects/e769006af8b34b1ea7cec029f337a840/users/945930196fe64d9dbd2d80be4eebba5d/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:17.357756 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc78ed-687d-43c7-a63f-ad3c20f31c85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e769006af8b34b1ea7cec029f337a840, user_id=945930196fe64d9dbd2d80be4eebba5d, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:17.363295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc78ed-687d-43c7-a63f-ad3c20f31c85 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:17.382209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1265/2524] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:17 2022] PUT /identity/v3/projects/e769006af8b34b1ea7cec029f337a840/users/945930196fe64d9dbd2d80be4eebba5d/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:17.386326 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74dc04b-df5c-45a0-ae88-a8792478a539 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:17.397405 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74dc04b-df5c-45a0-ae88-a8792478a539 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:17.449293 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:17.471199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cb00f-30f8-453b-83e0-2cbe80d7b2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:17.471811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cb00f-30f8-453b-83e0-2cbe80d7b2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:17.472108 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cb00f-30f8-453b-83e0-2cbe80d7b2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:17.472384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cb00f-30f8-453b-83e0-2cbe80d7b2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:17.473010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cb00f-30f8-453b-83e0-2cbe80d7b2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:17.474714 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cb00f-30f8-453b-83e0-2cbe80d7b2f1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:17.485642 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1260/2525] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:17.489912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc78ed-687d-43c7-a63f-ad3c20f31c85 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:17.500870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc78ed-687d-43c7-a63f-ad3c20f31c85 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:17.505723 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad504dbe-337a-48fe-b12c-5ef141f76ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:17.506396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad504dbe-337a-48fe-b12c-5ef141f76ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:17.506673 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad504dbe-337a-48fe-b12c-5ef141f76ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:17.506948 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad504dbe-337a-48fe-b12c-5ef141f76ef0 admin admin] PATH_INFO: `/v3/projects/e769006af8b34b1ea7cec029f337a840/users/945930196fe64d9dbd2d80be4eebba5d/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:17.508933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad504dbe-337a-48fe-b12c-5ef141f76ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e769006af8b34b1ea7cec029f337a840, user_id=945930196fe64d9dbd2d80be4eebba5d, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:17.511559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad504dbe-337a-48fe-b12c-5ef141f76ef0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:17.523530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1266/2526] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:17 2022] PUT /identity/v3/projects/e769006af8b34b1ea7cec029f337a840/users/945930196fe64d9dbd2d80be4eebba5d/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:17.534967 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377031f-c195-41d5-af9e-87d7a9608ecc None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:17.535192 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377031f-c195-41d5-af9e-87d7a9608ecc None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:17.535466 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377031f-c195-41d5-af9e-87d7a9608ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:17.569067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e377031f-c195-41d5-af9e-87d7a9608ecc None None] MFA Rules not processed for user `945930196fe64d9dbd2d80be4eebba5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:17.570321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e377031f-c195-41d5-af9e-87d7a9608ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:17.642916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1261/2527] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:17.654454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad504dbe-337a-48fe-b12c-5ef141f76ef0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:17.656277 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ad504dbe-337a-48fe-b12c-5ef141f76ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:17.714273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad504dbe-337a-48fe-b12c-5ef141f76ef0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:17.719419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad807409-0017-47db-8b84-2022b1dd5c5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:17.720121 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad807409-0017-47db-8b84-2022b1dd5c5b service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:17.720407 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad807409-0017-47db-8b84-2022b1dd5c5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:17.720684 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad807409-0017-47db-8b84-2022b1dd5c5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:17.792645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad807409-0017-47db-8b84-2022b1dd5c5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:17.793994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad807409-0017-47db-8b84-2022b1dd5c5b service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:17.807467 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1267/2528] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:22.124612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377031f-c195-41d5-af9e-87d7a9608ecc None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:22.134257 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377031f-c195-41d5-af9e-87d7a9608ecc None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:22.138015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba733265-b7f7-4c21-a020-cb56b5c6df64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:22.138645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba733265-b7f7-4c21-a020-cb56b5c6df64 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:22.138918 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba733265-b7f7-4c21-a020-cb56b5c6df64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:22.139188 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba733265-b7f7-4c21-a020-cb56b5c6df64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:22.148678 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba733265-b7f7-4c21-a020-cb56b5c6df64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:22.150097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba733265-b7f7-4c21-a020-cb56b5c6df64 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:22.159972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1262/2529] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:27.083485 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad807409-0017-47db-8b84-2022b1dd5c5b service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:27.119306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:27.153363 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad807409-0017-47db-8b84-2022b1dd5c5b service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:27.158724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9f327-47fb-4034-b6c0-f371a4e32d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:27.159346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9f327-47fb-4034-b6c0-f371a4e32d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:27.159624 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9f327-47fb-4034-b6c0-f371a4e32d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:27.159904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9f327-47fb-4034-b6c0-f371a4e32d8a admin admin] PATH_INFO: `/v3/users/4d4686c1d2984c6b9f738f9e1e75cdcb` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:27.161049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9f327-47fb-4034-b6c0-f371a4e32d8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d4686c1d2984c6b9f738f9e1e75cdcb)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:27.162963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9f327-47fb-4034-b6c0-f371a4e32d8a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:27.211384 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ccc9f327-47fb-4034-b6c0-f371a4e32d8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d4686c1d2984c6b9f738f9e1e75cdcb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:27.225438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ccc9f327-47fb-4034-b6c0-f371a4e32d8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d4686c1d2984c6b9f738f9e1e75cdcb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:27.232981 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ccc9f327-47fb-4034-b6c0-f371a4e32d8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d4686c1d2984c6b9f738f9e1e75cdcb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:27.234407 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ccc9f327-47fb-4034-b6c0-f371a4e32d8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d4686c1d2984c6b9f738f9e1e75cdcb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:27.241401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ccc9f327-47fb-4034-b6c0-f371a4e32d8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d4686c1d2984c6b9f738f9e1e75cdcb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:27.247367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1268/2530] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:27 2022] DELETE /identity/v3/users/4d4686c1d2984c6b9f738f9e1e75cdcb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:27.251859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cec48-d588-4e73-b6d9-904501f4b3ec None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:27.252366 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cec48-d588-4e73-b6d9-904501f4b3ec None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:27.252480 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cec48-d588-4e73-b6d9-904501f4b3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:27.271295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d03cec48-d588-4e73-b6d9-904501f4b3ec None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:27.272529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d03cec48-d588-4e73-b6d9-904501f4b3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:27.334483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1263/2531] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:27.344896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9f327-47fb-4034-b6c0-f371a4e32d8a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:27.346407 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ccc9f327-47fb-4034-b6c0-f371a4e32d8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:27.406369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9f327-47fb-4034-b6c0-f371a4e32d8a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:27.410002 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f75dc-a18d-4f90-a7d1-0ec757d56162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:27.410651 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f75dc-a18d-4f90-a7d1-0ec757d56162 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:27.410923 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f75dc-a18d-4f90-a7d1-0ec757d56162 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:27.411190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f75dc-a18d-4f90-a7d1-0ec757d56162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:27.420133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f75dc-a18d-4f90-a7d1-0ec757d56162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:27.421599 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f75dc-a18d-4f90-a7d1-0ec757d56162 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:27.431019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1269/2532] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:27.707821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03cec48-d588-4e73-b6d9-904501f4b3ec None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:27.709307 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d03cec48-d588-4e73-b6d9-904501f4b3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:27.723701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03cec48-d588-4e73-b6d9-904501f4b3ec None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:27.728414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:27.729041 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:27.729945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:27.730228 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c admin admin] PATH_INFO: `/v3/projects/53f46cbeb6ad4dd58d0df45d9487c88a` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:27.731521 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c admin admin] RBAC: Authorizing `identity:delete_project(project_id=53f46cbeb6ad4dd58d0df45d9487c88a)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:27.734129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:27.783297 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53f46cbeb6ad4dd58d0df45d9487c88a', 'request_id': 'req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:27.788743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53f46cbeb6ad4dd58d0df45d9487c88a', 'request_id': 'req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:27.790562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:27.797117 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53f46cbeb6ad4dd58d0df45d9487c88a', 'request_id': 'req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:27.809720 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c admin admin] The token cache is being invalidate because project 53f46cbeb6ad4dd58d0df45d9487c88a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:27.810008 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:27.810622 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:27.812508 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1264/2533] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:27 2022] DELETE /identity/v3/projects/53f46cbeb6ad4dd58d0df45d9487c88a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:27.816625 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f75dc-a18d-4f90-a7d1-0ec757d56162 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:27.826241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f75dc-a18d-4f90-a7d1-0ec757d56162 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:27.881877 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:27.891138 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a97e8-ec93-4752-bbee-276062321dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:27.892114 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a97e8-ec93-4752-bbee-276062321dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:27.892441 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a97e8-ec93-4752-bbee-276062321dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:27.892950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a97e8-ec93-4752-bbee-276062321dae admin admin] PATH_INFO: `/v3/users/945930196fe64d9dbd2d80be4eebba5d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:27.894550 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a97e8-ec93-4752-bbee-276062321dae admin admin] RBAC: Authorizing `identity:delete_user(user_id=945930196fe64d9dbd2d80be4eebba5d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:27.896497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a97e8-ec93-4752-bbee-276062321dae admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:27.951029 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c75a97e8-ec93-4752-bbee-276062321dae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '945930196fe64d9dbd2d80be4eebba5d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:27.964955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c75a97e8-ec93-4752-bbee-276062321dae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '945930196fe64d9dbd2d80be4eebba5d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:27.973495 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c75a97e8-ec93-4752-bbee-276062321dae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '945930196fe64d9dbd2d80be4eebba5d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:27.974054 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c75a97e8-ec93-4752-bbee-276062321dae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '945930196fe64d9dbd2d80be4eebba5d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:27.984091 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c75a97e8-ec93-4752-bbee-276062321dae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '945930196fe64d9dbd2d80be4eebba5d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:27.994055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1270/2534] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:27 2022] DELETE /identity/v3/users/945930196fe64d9dbd2d80be4eebba5d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:28.294109 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:28.302874 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:28.341732 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609f765-c4bf-4abb-a7fb-60c2c2bfc20c admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:28.414061 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf1cb2-529c-4b0c-bcd6-69a7a5c71075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:28.414610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf1cb2-529c-4b0c-bcd6-69a7a5c71075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:28.414884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf1cb2-529c-4b0c-bcd6-69a7a5c71075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:28.415159 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf1cb2-529c-4b0c-bcd6-69a7a5c71075 admin admin] PATH_INFO: `/v3/projects/e769006af8b34b1ea7cec029f337a840` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:28.416298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf1cb2-529c-4b0c-bcd6-69a7a5c71075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e769006af8b34b1ea7cec029f337a840)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:28.418197 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf1cb2-529c-4b0c-bcd6-69a7a5c71075 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:28.467911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-56cf1cb2-529c-4b0c-bcd6-69a7a5c71075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e769006af8b34b1ea7cec029f337a840', 'request_id': 'req-56cf1cb2-529c-4b0c-bcd6-69a7a5c71075'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:28.473933 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-56cf1cb2-529c-4b0c-bcd6-69a7a5c71075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e769006af8b34b1ea7cec029f337a840', 'request_id': 'req-56cf1cb2-529c-4b0c-bcd6-69a7a5c71075'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:28.480158 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-56cf1cb2-529c-4b0c-bcd6-69a7a5c71075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e769006af8b34b1ea7cec029f337a840', 'request_id': 'req-56cf1cb2-529c-4b0c-bcd6-69a7a5c71075'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:28.493498 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-56cf1cb2-529c-4b0c-bcd6-69a7a5c71075 admin admin] The token cache is being invalidate because project e769006af8b34b1ea7cec029f337a840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:28.493786 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-56cf1cb2-529c-4b0c-bcd6-69a7a5c71075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:28.494402 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-56cf1cb2-529c-4b0c-bcd6-69a7a5c71075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:28.495878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1265/2535] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:28 2022] DELETE /identity/v3/projects/e769006af8b34b1ea7cec029f337a840 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:28.502056 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ebdb4-a7fd-4e99-98da-3ad44be1ffd3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:28.503085 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ebdb4-a7fd-4e99-98da-3ad44be1ffd3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:28.503433 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ebdb4-a7fd-4e99-98da-3ad44be1ffd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:28.525045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-361ebdb4-a7fd-4e99-98da-3ad44be1ffd3 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:28.527551 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-361ebdb4-a7fd-4e99-98da-3ad44be1ffd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:28.609788 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1271/2536] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:28.618358 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95f323-6ff7-481f-913a-e220f56e6cc2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:28.618641 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95f323-6ff7-481f-913a-e220f56e6cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:28.618917 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95f323-6ff7-481f-913a-e220f56e6cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:28.645294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9c95f323-6ff7-481f-913a-e220f56e6cc2 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:28.646495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9c95f323-6ff7-481f-913a-e220f56e6cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:28.659455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1266/2537] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:28.662198 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ebdb4-a7fd-4e99-98da-3ad44be1ffd3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:28.673953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ebdb4-a7fd-4e99-98da-3ad44be1ffd3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:28.679031 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c543de-c89a-4e36-aecd-b218a73fb928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:28.679649 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c543de-c89a-4e36-aecd-b218a73fb928 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:28.679925 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c543de-c89a-4e36-aecd-b218a73fb928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:28.680263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c543de-c89a-4e36-aecd-b218a73fb928 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:28.680884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c543de-c89a-4e36-aecd-b218a73fb928 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:02:28.681185 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c543de-c89a-4e36-aecd-b218a73fb928 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:28.682927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c543de-c89a-4e36-aecd-b218a73fb928 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:28.702785 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1272/2538] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 08:02:28.707588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95f323-6ff7-481f-913a-e220f56e6cc2 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:28.722076 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95f323-6ff7-481f-913a-e220f56e6cc2 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:28.730358 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb04cb3-492f-4386-afdf-2b81770062a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:28.731218 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb04cb3-492f-4386-afdf-2b81770062a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:28.731493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb04cb3-492f-4386-afdf-2b81770062a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:28.731757 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb04cb3-492f-4386-afdf-2b81770062a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:28.732461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb04cb3-492f-4386-afdf-2b81770062a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:28.735498 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb04cb3-492f-4386-afdf-2b81770062a1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:28.779154 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1267/2539] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:28 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:28.784274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c543de-c89a-4e36-aecd-b218a73fb928 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:28.797223 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c543de-c89a-4e36-aecd-b218a73fb928 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:28.871050 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f71e60-7eb8-418b-8c45-4c0cbdd4bc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:28.871775 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f71e60-7eb8-418b-8c45-4c0cbdd4bc50 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:28.872157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f71e60-7eb8-418b-8c45-4c0cbdd4bc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:28.872437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f71e60-7eb8-418b-8c45-4c0cbdd4bc50 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:28.874521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f71e60-7eb8-418b-8c45-4c0cbdd4bc50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:28.876334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f71e60-7eb8-418b-8c45-4c0cbdd4bc50 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:28.886331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:28.904366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1273/2540] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:28 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:28.907430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb04cb3-492f-4386-afdf-2b81770062a1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:28.920138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb04cb3-492f-4386-afdf-2b81770062a1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:28.926268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec12054-c5f9-43eb-9970-5379ae21857d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:28.926971 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec12054-c5f9-43eb-9970-5379ae21857d admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:28.927312 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec12054-c5f9-43eb-9970-5379ae21857d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:28.927647 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec12054-c5f9-43eb-9970-5379ae21857d admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:28.928391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec12054-c5f9-43eb-9970-5379ae21857d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:28.930223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec12054-c5f9-43eb-9970-5379ae21857d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:28.943130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1268/2541] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:28.946983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f71e60-7eb8-418b-8c45-4c0cbdd4bc50 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:28.960464 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f71e60-7eb8-418b-8c45-4c0cbdd4bc50 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:28.966166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c1b58-9040-4273-8155-94b5574200a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:28.966853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c1b58-9040-4273-8155-94b5574200a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:28.967236 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c1b58-9040-4273-8155-94b5574200a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:28.968229 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c1b58-9040-4273-8155-94b5574200a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:28.969194 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c1b58-9040-4273-8155-94b5574200a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:28.970862 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c1b58-9040-4273-8155-94b5574200a3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:28.984991 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1274/2542] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:28.990049 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec12054-c5f9-43eb-9970-5379ae21857d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:29.001142 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec12054-c5f9-43eb-9970-5379ae21857d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:29.005790 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4141f0c-7ad0-4217-96fd-96dd3484cb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:29.006439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4141f0c-7ad0-4217-96fd-96dd3484cb76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:29.006712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4141f0c-7ad0-4217-96fd-96dd3484cb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:29.006981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4141f0c-7ad0-4217-96fd-96dd3484cb76 admin admin] PATH_INFO: `/v3/projects/b7f7a616ffc84d23af3691a2672a6829/users/7a67efdb6af84cfb92df1be6d8345ea9/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:29.025657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4141f0c-7ad0-4217-96fd-96dd3484cb76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7f7a616ffc84d23af3691a2672a6829, user_id=7a67efdb6af84cfb92df1be6d8345ea9, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:29.028469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4141f0c-7ad0-4217-96fd-96dd3484cb76 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:29.044603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1269/2543] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:28 2022] PUT /identity/v3/projects/b7f7a616ffc84d23af3691a2672a6829/users/7a67efdb6af84cfb92df1be6d8345ea9/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:29.051556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433bfbb-4f5d-4bd5-b942-f19b4a310900 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:29.053147 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433bfbb-4f5d-4bd5-b942-f19b4a310900 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:29.053649 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433bfbb-4f5d-4bd5-b942-f19b4a310900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:29.079568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c433bfbb-4f5d-4bd5-b942-f19b4a310900 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:29.081075 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c433bfbb-4f5d-4bd5-b942-f19b4a310900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:29.147601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1275/2544] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:29.157607 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4141f0c-7ad0-4217-96fd-96dd3484cb76 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:29.159597 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c4141f0c-7ad0-4217-96fd-96dd3484cb76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:29.187303 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:29.230157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4141f0c-7ad0-4217-96fd-96dd3484cb76 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:29.234487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa177bc-884a-4963-8256-c21835ed1aa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:29.235096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa177bc-884a-4963-8256-c21835ed1aa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:29.235374 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa177bc-884a-4963-8256-c21835ed1aa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:29.235646 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa177bc-884a-4963-8256-c21835ed1aa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:29.245024 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa177bc-884a-4963-8256-c21835ed1aa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:29.246617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa177bc-884a-4963-8256-c21835ed1aa4 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:29.256985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1270/2545] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:02:30.447052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7c8ff-d8e6-4df7-af9d-1421a1decb6d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:30.447583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7c8ff-d8e6-4df7-af9d-1421a1decb6d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:30.447666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7c8ff-d8e6-4df7-af9d-1421a1decb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:30.501263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5ee7c8ff-d8e6-4df7-af9d-1421a1decb6d None None] MFA Rules not processed for user `7a67efdb6af84cfb92df1be6d8345ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:30.502915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5ee7c8ff-d8e6-4df7-af9d-1421a1decb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:30.599071 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1276/2546] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:30.608069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa177bc-884a-4963-8256-c21835ed1aa4 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:30.610339 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7aa177bc-884a-4963-8256-c21835ed1aa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:30.712252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa177bc-884a-4963-8256-c21835ed1aa4 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:30.717476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b1e9a-5bf4-4d51-8931-827df426e2a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:30.718211 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b1e9a-5bf4-4d51-8931-827df426e2a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:30.718556 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b1e9a-5bf4-4d51-8931-827df426e2a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:30.718946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b1e9a-5bf4-4d51-8931-827df426e2a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:30.727859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b1e9a-5bf4-4d51-8931-827df426e2a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:30.729351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b1e9a-5bf4-4d51-8931-827df426e2a2 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:30.738476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1271/2547] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:32.205764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cb4b4-b65f-435d-b7f8-3b2c923dde9b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:32.206228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cb4b4-b65f-435d-b7f8-3b2c923dde9b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:32.206362 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cb4b4-b65f-435d-b7f8-3b2c923dde9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:32.229207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7d1cb4b4-b65f-435d-b7f8-3b2c923dde9b None None] MFA Rules not processed for user `7a67efdb6af84cfb92df1be6d8345ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:32.231318 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7d1cb4b4-b65f-435d-b7f8-3b2c923dde9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:32.239620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1277/2548] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:34.628370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b1e9a-5bf4-4d51-8931-827df426e2a2 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:34.629787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2e5b1e9a-5bf4-4d51-8931-827df426e2a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:34.642599 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b1e9a-5bf4-4d51-8931-827df426e2a2 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:34.647744 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815337f6-5745-4f50-afb7-785c37010b09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:34.648368 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815337f6-5745-4f50-afb7-785c37010b09 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:34.648646 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815337f6-5745-4f50-afb7-785c37010b09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:34.648914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815337f6-5745-4f50-afb7-785c37010b09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:34.651130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-815337f6-5745-4f50-afb7-785c37010b09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:34.665237 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815337f6-5745-4f50-afb7-785c37010b09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:34.666682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815337f6-5745-4f50-afb7-785c37010b09 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:34.677660 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1272/2549] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:35.422826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1cb4b4-b65f-435d-b7f8-3b2c923dde9b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:35.424305 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7d1cb4b4-b65f-435d-b7f8-3b2c923dde9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:35.438602 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1cb4b4-b65f-435d-b7f8-3b2c923dde9b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:35.443776 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b6cbd-96ea-40d1-945a-8e6be7cb3b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:35.444446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b6cbd-96ea-40d1-945a-8e6be7cb3b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:35.444725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b6cbd-96ea-40d1-945a-8e6be7cb3b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:35.445000 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b6cbd-96ea-40d1-945a-8e6be7cb3b01 admin admin] PATH_INFO: `/v3/users/a3eeabb2bed246c9b616360a5965cc23` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:35.446126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b6cbd-96ea-40d1-945a-8e6be7cb3b01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3eeabb2bed246c9b616360a5965cc23)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:35.448036 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b6cbd-96ea-40d1-945a-8e6be7cb3b01 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:35.501968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c79b6cbd-96ea-40d1-945a-8e6be7cb3b01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3eeabb2bed246c9b616360a5965cc23'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:35.509882 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:INSERT=2,keystone:SELECT=90,keystone:DELETE=8 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:35.515956 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c79b6cbd-96ea-40d1-945a-8e6be7cb3b01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3eeabb2bed246c9b616360a5965cc23'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:35.524317 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c79b6cbd-96ea-40d1-945a-8e6be7cb3b01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3eeabb2bed246c9b616360a5965cc23'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:35.524941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c79b6cbd-96ea-40d1-945a-8e6be7cb3b01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3eeabb2bed246c9b616360a5965cc23'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:35.533677 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c79b6cbd-96ea-40d1-945a-8e6be7cb3b01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3eeabb2bed246c9b616360a5965cc23'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:35.540306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1278/2550] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:35 2022] DELETE /identity/v3/users/a3eeabb2bed246c9b616360a5965cc23 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:35.544684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0bf7d-f6cb-4e94-932b-4198b40bb9a4 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:35.544972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0bf7d-f6cb-4e94-932b-4198b40bb9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:35.545263 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0bf7d-f6cb-4e94-932b-4198b40bb9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:35.566487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-99d0bf7d-f6cb-4e94-932b-4198b40bb9a4 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:35.567709 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-99d0bf7d-f6cb-4e94-932b-4198b40bb9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:35.642812 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1273/2551] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:35.652661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b6cbd-96ea-40d1-945a-8e6be7cb3b01 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:35.655430 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c79b6cbd-96ea-40d1-945a-8e6be7cb3b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:35.725702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b6cbd-96ea-40d1-945a-8e6be7cb3b01 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:35.729707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15675525-5c73-42fe-a95e-c0dece9dc5bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:35.730432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15675525-5c73-42fe-a95e-c0dece9dc5bd service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:35.730831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15675525-5c73-42fe-a95e-c0dece9dc5bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:35.731200 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15675525-5c73-42fe-a95e-c0dece9dc5bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:35.744143 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15675525-5c73-42fe-a95e-c0dece9dc5bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:35.745671 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15675525-5c73-42fe-a95e-c0dece9dc5bd service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:35.761125 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1279/2552] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:36.317517 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0bf7d-f6cb-4e94-932b-4198b40bb9a4 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:36.319072 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-99d0bf7d-f6cb-4e94-932b-4198b40bb9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:36.337278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0bf7d-f6cb-4e94-932b-4198b40bb9a4 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:36.344654 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f7d90-6471-484c-9747-df9bd9be01ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:36.345965 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f7d90-6471-484c-9747-df9bd9be01ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:36.353627 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f7d90-6471-484c-9747-df9bd9be01ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:36.354120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f7d90-6471-484c-9747-df9bd9be01ad admin admin] PATH_INFO: `/v3/projects/207fcea03c5d45b3b4782b66ae52f038` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:36.355410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f7d90-6471-484c-9747-df9bd9be01ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=207fcea03c5d45b3b4782b66ae52f038)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:36.357393 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f7d90-6471-484c-9747-df9bd9be01ad admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:36.408477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:36.416125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-437f7d90-6471-484c-9747-df9bd9be01ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '207fcea03c5d45b3b4782b66ae52f038', 'request_id': 'req-437f7d90-6471-484c-9747-df9bd9be01ad'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:36.421865 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-437f7d90-6471-484c-9747-df9bd9be01ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '207fcea03c5d45b3b4782b66ae52f038', 'request_id': 'req-437f7d90-6471-484c-9747-df9bd9be01ad'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:36.428181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-437f7d90-6471-484c-9747-df9bd9be01ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '207fcea03c5d45b3b4782b66ae52f038', 'request_id': 'req-437f7d90-6471-484c-9747-df9bd9be01ad'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:36.441895 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-437f7d90-6471-484c-9747-df9bd9be01ad admin admin] The token cache is being invalidate because project 207fcea03c5d45b3b4782b66ae52f038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:36.442188 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-437f7d90-6471-484c-9747-df9bd9be01ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:36.442823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-437f7d90-6471-484c-9747-df9bd9be01ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:36.444359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1274/2553] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:36 2022] DELETE /identity/v3/projects/207fcea03c5d45b3b4782b66ae52f038 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:36.449163 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aca76-816c-4e84-8e1d-0155fff7bfa9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:36.449633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aca76-816c-4e84-8e1d-0155fff7bfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:36.449751 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aca76-816c-4e84-8e1d-0155fff7bfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:36.469562 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f38aca76-816c-4e84-8e1d-0155fff7bfa9 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:36.470701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f38aca76-816c-4e84-8e1d-0155fff7bfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:36.537696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1280/2554] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:36.546216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae045dd7-ba34-40bb-9d70-9ba157a831f7 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:36.546521 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae045dd7-ba34-40bb-9d70-9ba157a831f7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:36.546792 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae045dd7-ba34-40bb-9d70-9ba157a831f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:36.569297 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ae045dd7-ba34-40bb-9d70-9ba157a831f7 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:36.570500 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ae045dd7-ba34-40bb-9d70-9ba157a831f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:36.578396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1275/2555] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:36.582845 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38aca76-816c-4e84-8e1d-0155fff7bfa9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:36.605450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38aca76-816c-4e84-8e1d-0155fff7bfa9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:36.612817 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8b13e-483e-4d03-8b26-9ebd39d3112d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:36.613886 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8b13e-483e-4d03-8b26-9ebd39d3112d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:36.614167 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8b13e-483e-4d03-8b26-9ebd39d3112d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:36.614442 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8b13e-483e-4d03-8b26-9ebd39d3112d admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:36.615331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8b13e-483e-4d03-8b26-9ebd39d3112d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:02:36.615457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8b13e-483e-4d03-8b26-9ebd39d3112d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:36.617482 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8b13e-483e-4d03-8b26-9ebd39d3112d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:36.630969 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1281/2556] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:02:36.635537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae045dd7-ba34-40bb-9d70-9ba157a831f7 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:36.645279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae045dd7-ba34-40bb-9d70-9ba157a831f7 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:36.649981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd291d-e438-416e-b989-471b3485c6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:36.650584 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd291d-e438-416e-b989-471b3485c6bc admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:36.650858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd291d-e438-416e-b989-471b3485c6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:36.651127 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd291d-e438-416e-b989-471b3485c6bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:36.651748 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd291d-e438-416e-b989-471b3485c6bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:36.653673 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd291d-e438-416e-b989-471b3485c6bc admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:36.684986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1276/2557] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:36 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:36.687218 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8b13e-483e-4d03-8b26-9ebd39d3112d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:36.697192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8b13e-483e-4d03-8b26-9ebd39d3112d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:36.755260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400caa91-2e6e-4f0e-9d4f-1ac6c056f37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:36.757398 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400caa91-2e6e-4f0e-9d4f-1ac6c056f37f admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:36.758019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400caa91-2e6e-4f0e-9d4f-1ac6c056f37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:36.758019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400caa91-2e6e-4f0e-9d4f-1ac6c056f37f admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:36.758606 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400caa91-2e6e-4f0e-9d4f-1ac6c056f37f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:36.760580 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400caa91-2e6e-4f0e-9d4f-1ac6c056f37f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:36.772567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:36.786422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1282/2558] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:36 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:36.791729 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd291d-e438-416e-b989-471b3485c6bc admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:36.803278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd291d-e438-416e-b989-471b3485c6bc admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:36.808260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a5773-bc93-4787-92be-cef21df92938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:36.808871 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a5773-bc93-4787-92be-cef21df92938 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:36.809149 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a5773-bc93-4787-92be-cef21df92938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:36.809418 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a5773-bc93-4787-92be-cef21df92938 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:36.810051 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a5773-bc93-4787-92be-cef21df92938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:36.812291 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a5773-bc93-4787-92be-cef21df92938 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:36.822755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1277/2559] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:36.826780 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400caa91-2e6e-4f0e-9d4f-1ac6c056f37f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:36.838344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400caa91-2e6e-4f0e-9d4f-1ac6c056f37f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:36.843641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e42f06-9036-4497-b8d1-ccdfa3c5caa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:36.844278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e42f06-9036-4497-b8d1-ccdfa3c5caa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:36.844559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e42f06-9036-4497-b8d1-ccdfa3c5caa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:36.844838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e42f06-9036-4497-b8d1-ccdfa3c5caa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:36.845486 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e42f06-9036-4497-b8d1-ccdfa3c5caa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:36.847326 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e42f06-9036-4497-b8d1-ccdfa3c5caa0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:36.860896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1283/2560] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:36.865456 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a5773-bc93-4787-92be-cef21df92938 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:36.876637 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a5773-bc93-4787-92be-cef21df92938 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:36.881922 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ebb8b-2eaf-4c63-a85f-a5574508c64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:36.882703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ebb8b-2eaf-4c63-a85f-a5574508c64f admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:36.882988 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ebb8b-2eaf-4c63-a85f-a5574508c64f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:36.883258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ebb8b-2eaf-4c63-a85f-a5574508c64f admin admin] PATH_INFO: `/v3/projects/a6b39e753f5847b295dbfd9169898345/users/3c52c67da8164dc1ae9c62979645fe21/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:36.912225 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ebb8b-2eaf-4c63-a85f-a5574508c64f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b39e753f5847b295dbfd9169898345, user_id=3c52c67da8164dc1ae9c62979645fe21, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:36.915398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ebb8b-2eaf-4c63-a85f-a5574508c64f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:36.941076 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1278/2561] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:36 2022] PUT /identity/v3/projects/a6b39e753f5847b295dbfd9169898345/users/3c52c67da8164dc1ae9c62979645fe21/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:36.953241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c33168-780c-4828-bb46-2577acae8f98 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:36.953607 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c33168-780c-4828-bb46-2577acae8f98 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:36.953961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c33168-780c-4828-bb46-2577acae8f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:37.000260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-40c33168-780c-4828-bb46-2577acae8f98 None None] MFA Rules not processed for user `3c52c67da8164dc1ae9c62979645fe21`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:37.001619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-40c33168-780c-4828-bb46-2577acae8f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:37.105600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1284/2562] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:37.107155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ebb8b-2eaf-4c63-a85f-a5574508c64f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:37.186596 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ebb8b-2eaf-4c63-a85f-a5574508c64f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:37.191743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02979eb-6240-483d-99d6-b21658542b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:37.192388 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02979eb-6240-483d-99d6-b21658542b1a admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:37.192690 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02979eb-6240-483d-99d6-b21658542b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:37.193426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02979eb-6240-483d-99d6-b21658542b1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:37.193780 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02979eb-6240-483d-99d6-b21658542b1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:37.195771 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02979eb-6240-483d-99d6-b21658542b1a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:37.226907 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1279/2563] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:37 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:37.228237 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c33168-780c-4828-bb46-2577acae8f98 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:37.239693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c33168-780c-4828-bb46-2577acae8f98 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:37.323768 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f52db-89db-4176-bdc5-8cff49d66840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:37.324507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f52db-89db-4176-bdc5-8cff49d66840 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:37.324856 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f52db-89db-4176-bdc5-8cff49d66840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:37.325199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f52db-89db-4176-bdc5-8cff49d66840 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:37.327244 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f52db-89db-4176-bdc5-8cff49d66840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:37.329662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f52db-89db-4176-bdc5-8cff49d66840 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:37.381035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1285/2564] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:37 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:37.386438 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02979eb-6240-483d-99d6-b21658542b1a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:37.412492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02979eb-6240-483d-99d6-b21658542b1a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:37.423200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d683c84-997f-4e7d-be97-5f0df908bf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:37.424325 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d683c84-997f-4e7d-be97-5f0df908bf11 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:37.424698 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d683c84-997f-4e7d-be97-5f0df908bf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:37.425169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d683c84-997f-4e7d-be97-5f0df908bf11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:37.425953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d683c84-997f-4e7d-be97-5f0df908bf11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:37.427579 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d683c84-997f-4e7d-be97-5f0df908bf11 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:37.443356 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1280/2565] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:37.447357 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f52db-89db-4176-bdc5-8cff49d66840 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:37.458500 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f52db-89db-4176-bdc5-8cff49d66840 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:37.463502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc6224-6964-4b9f-9264-7d0d16843992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:37.464340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc6224-6964-4b9f-9264-7d0d16843992 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:37.465353 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc6224-6964-4b9f-9264-7d0d16843992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:37.465353 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc6224-6964-4b9f-9264-7d0d16843992 admin admin] PATH_INFO: `/v3/projects/b19e0992dac048438225ba6acaa9c873/users/4698e65ceb664749a89260343bdbec0d/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:37.482703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc6224-6964-4b9f-9264-7d0d16843992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19e0992dac048438225ba6acaa9c873, user_id=4698e65ceb664749a89260343bdbec0d, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:37.486594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc6224-6964-4b9f-9264-7d0d16843992 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:37.506437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1286/2566] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:37 2022] PUT /identity/v3/projects/b19e0992dac048438225ba6acaa9c873/users/4698e65ceb664749a89260343bdbec0d/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:37.507553 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d683c84-997f-4e7d-be97-5f0df908bf11 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:37.520401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d683c84-997f-4e7d-be97-5f0df908bf11 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:37.578221 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:37.620289 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d915a-9829-4f86-bac1-5653fe12819f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:37.620991 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d915a-9829-4f86-bac1-5653fe12819f admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:37.621331 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d915a-9829-4f86-bac1-5653fe12819f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:37.621714 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d915a-9829-4f86-bac1-5653fe12819f admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:37.623085 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d915a-9829-4f86-bac1-5653fe12819f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:37.625630 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d915a-9829-4f86-bac1-5653fe12819f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:37.642071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1281/2567] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:37.646172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc6224-6964-4b9f-9264-7d0d16843992 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:37.658264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc6224-6964-4b9f-9264-7d0d16843992 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:37.663351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ebe599-4df4-4634-ab75-c0755db7b245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:37.664090 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebe599-4df4-4634-ab75-c0755db7b245 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:37.664386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebe599-4df4-4634-ab75-c0755db7b245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:37.664659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebe599-4df4-4634-ab75-c0755db7b245 admin admin] PATH_INFO: `/v3/projects/b19e0992dac048438225ba6acaa9c873/users/4698e65ceb664749a89260343bdbec0d/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:37.667581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ebe599-4df4-4634-ab75-c0755db7b245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19e0992dac048438225ba6acaa9c873, user_id=4698e65ceb664749a89260343bdbec0d, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:37.669906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ebe599-4df4-4634-ab75-c0755db7b245 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:37.692627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1287/2568] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:37 2022] PUT /identity/v3/projects/b19e0992dac048438225ba6acaa9c873/users/4698e65ceb664749a89260343bdbec0d/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:37.695669 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e9b71-324b-4b69-ada2-5a4ba51f60b3 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:37.696595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e9b71-324b-4b69-ada2-5a4ba51f60b3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:37.697037 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e9b71-324b-4b69-ada2-5a4ba51f60b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:37.741437 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-358e9b71-324b-4b69-ada2-5a4ba51f60b3 None None] MFA Rules not processed for user `4698e65ceb664749a89260343bdbec0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:37.743241 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-358e9b71-324b-4b69-ada2-5a4ba51f60b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:37.837217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1282/2569] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:37.850179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ebe599-4df4-4634-ab75-c0755db7b245 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:37.851958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c5ebe599-4df4-4634-ab75-c0755db7b245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:37.896493 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:37.940603 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ebe599-4df4-4634-ab75-c0755db7b245 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:37.946405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085667b-43c0-4aa7-8a48-0aa9e5144c46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:37.947181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085667b-43c0-4aa7-8a48-0aa9e5144c46 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:37.947467 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085667b-43c0-4aa7-8a48-0aa9e5144c46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:37.947806 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085667b-43c0-4aa7-8a48-0aa9e5144c46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:38.027794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085667b-43c0-4aa7-8a48-0aa9e5144c46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:38.029342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085667b-43c0-4aa7-8a48-0aa9e5144c46 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:38.044564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1288/2570] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:02:40.954172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358e9b71-324b-4b69-ada2-5a4ba51f60b3 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:40.993818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085667b-43c0-4aa7-8a48-0aa9e5144c46 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:40.995791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c085667b-43c0-4aa7-8a48-0aa9e5144c46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:41.042970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358e9b71-324b-4b69-ada2-5a4ba51f60b3 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:41.048350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57df0d-b975-4fef-a169-efba198b82d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:41.048978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57df0d-b975-4fef-a169-efba198b82d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:41.049258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57df0d-b975-4fef-a169-efba198b82d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:41.049522 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57df0d-b975-4fef-a169-efba198b82d7 admin admin] PATH_INFO: `/v3/users/3c52c67da8164dc1ae9c62979645fe21` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:41.050742 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57df0d-b975-4fef-a169-efba198b82d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c52c67da8164dc1ae9c62979645fe21)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:41.053569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57df0d-b975-4fef-a169-efba198b82d7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:41.077549 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085667b-43c0-4aa7-8a48-0aa9e5144c46 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:41.083209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac20c5b-4a4e-451d-9fee-6a3c71b90f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:41.083871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac20c5b-4a4e-451d-9fee-6a3c71b90f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:41.084180 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac20c5b-4a4e-451d-9fee-6a3c71b90f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:41.084455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac20c5b-4a4e-451d-9fee-6a3c71b90f85 admin admin] PATH_INFO: `/v3/users/457d69a7306d46cd8445f887eb95c517` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:41.085606 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac20c5b-4a4e-451d-9fee-6a3c71b90f85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=457d69a7306d46cd8445f887eb95c517)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:41.087810 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac20c5b-4a4e-451d-9fee-6a3c71b90f85 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:41.117773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9d57df0d-b975-4fef-a169-efba198b82d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c52c67da8164dc1ae9c62979645fe21'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.118512 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9d57df0d-b975-4fef-a169-efba198b82d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c52c67da8164dc1ae9c62979645fe21'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.133499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9d57df0d-b975-4fef-a169-efba198b82d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c52c67da8164dc1ae9c62979645fe21'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.134246 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9d57df0d-b975-4fef-a169-efba198b82d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c52c67da8164dc1ae9c62979645fe21'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.135805 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:41.146570 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4ac20c5b-4a4e-451d-9fee-6a3c71b90f85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '457d69a7306d46cd8445f887eb95c517'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.148919 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9d57df0d-b975-4fef-a169-efba198b82d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c52c67da8164dc1ae9c62979645fe21'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.162185 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4ac20c5b-4a4e-451d-9fee-6a3c71b90f85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '457d69a7306d46cd8445f887eb95c517'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.165395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1283/2571] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:40 2022] DELETE /identity/v3/users/3c52c67da8164dc1ae9c62979645fe21 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:41.169897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f70ef-1b00-4b0d-aaa9-042687d1b972 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:41.170439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f70ef-1b00-4b0d-aaa9-042687d1b972 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:41.170990 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f70ef-1b00-4b0d-aaa9-042687d1b972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:41.173942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4ac20c5b-4a4e-451d-9fee-6a3c71b90f85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '457d69a7306d46cd8445f887eb95c517'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.174678 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4ac20c5b-4a4e-451d-9fee-6a3c71b90f85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '457d69a7306d46cd8445f887eb95c517'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.185380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4ac20c5b-4a4e-451d-9fee-6a3c71b90f85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '457d69a7306d46cd8445f887eb95c517'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.188391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:41.196923 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1289/2572] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:40 2022] DELETE /identity/v3/users/457d69a7306d46cd8445f887eb95c517 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:41.197058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-4a5f70ef-1b00-4b0d-aaa9-042687d1b972 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:41.198094 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4a5f70ef-1b00-4b0d-aaa9-042687d1b972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:41.262574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1284/2573] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:41.273057 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac20c5b-4a4e-451d-9fee-6a3c71b90f85 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:41.274508 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4ac20c5b-4a4e-451d-9fee-6a3c71b90f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:41.336559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac20c5b-4a4e-451d-9fee-6a3c71b90f85 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:41.340445 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabe772-a8ca-4bf2-a919-12e5e55a014d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:41.341058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabe772-a8ca-4bf2-a919-12e5e55a014d service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:41.341338 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabe772-a8ca-4bf2-a919-12e5e55a014d service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:41.341854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabe772-a8ca-4bf2-a919-12e5e55a014d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:41.357135 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aabe772-a8ca-4bf2-a919-12e5e55a014d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:41.359372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aabe772-a8ca-4bf2-a919-12e5e55a014d service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:41.371608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1290/2574] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:41.416126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f70ef-1b00-4b0d-aaa9-042687d1b972 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:41.416126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4a5f70ef-1b00-4b0d-aaa9-042687d1b972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:41.429482 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f70ef-1b00-4b0d-aaa9-042687d1b972 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:41.435040 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8578d3f-6748-4e96-8fa3-669ed62a7ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:41.438121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8578d3f-6748-4e96-8fa3-669ed62a7ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:41.438366 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8578d3f-6748-4e96-8fa3-669ed62a7ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:41.438617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8578d3f-6748-4e96-8fa3-669ed62a7ced admin admin] PATH_INFO: `/v3/projects/18463a34e4aa4b89a2a9ccad75a0d81e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:41.440210 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8578d3f-6748-4e96-8fa3-669ed62a7ced admin admin] RBAC: Authorizing `identity:delete_project(project_id=18463a34e4aa4b89a2a9ccad75a0d81e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:41.442196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8578d3f-6748-4e96-8fa3-669ed62a7ced admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:41.509016 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e8578d3f-6748-4e96-8fa3-669ed62a7ced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18463a34e4aa4b89a2a9ccad75a0d81e', 'request_id': 'req-e8578d3f-6748-4e96-8fa3-669ed62a7ced'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.516902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e8578d3f-6748-4e96-8fa3-669ed62a7ced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18463a34e4aa4b89a2a9ccad75a0d81e', 'request_id': 'req-e8578d3f-6748-4e96-8fa3-669ed62a7ced'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.524675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e8578d3f-6748-4e96-8fa3-669ed62a7ced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18463a34e4aa4b89a2a9ccad75a0d81e', 'request_id': 'req-e8578d3f-6748-4e96-8fa3-669ed62a7ced'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.541524 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e8578d3f-6748-4e96-8fa3-669ed62a7ced admin admin] The token cache is being invalidate because project 18463a34e4aa4b89a2a9ccad75a0d81e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:41.543506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e8578d3f-6748-4e96-8fa3-669ed62a7ced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.544370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e8578d3f-6748-4e96-8fa3-669ed62a7ced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.546992 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1285/2575] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:41 2022] DELETE /identity/v3/projects/18463a34e4aa4b89a2a9ccad75a0d81e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:41.551758 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64c25b-4387-4b9e-8db1-d851921a6338 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:41.552185 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64c25b-4387-4b9e-8db1-d851921a6338 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:41.552519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64c25b-4387-4b9e-8db1-d851921a6338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:41.575310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4a64c25b-4387-4b9e-8db1-d851921a6338 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:41.577370 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4a64c25b-4387-4b9e-8db1-d851921a6338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:41.657427 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1291/2576] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:41.664112 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a37b0c-8cec-4a43-b3cd-7f426561f61d None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:41.664396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a37b0c-8cec-4a43-b3cd-7f426561f61d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:41.664671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a37b0c-8cec-4a43-b3cd-7f426561f61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:41.689126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-47a37b0c-8cec-4a43-b3cd-7f426561f61d None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:41.690401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-47a37b0c-8cec-4a43-b3cd-7f426561f61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:41.700005 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1286/2577] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:41.704725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64c25b-4387-4b9e-8db1-d851921a6338 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:41.715518 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64c25b-4387-4b9e-8db1-d851921a6338 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:41.722163 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773967e-425f-4f8c-902a-293c5aa8f383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:41.723057 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773967e-425f-4f8c-902a-293c5aa8f383 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:41.723402 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773967e-425f-4f8c-902a-293c5aa8f383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:41.724215 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773967e-425f-4f8c-902a-293c5aa8f383 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:41.724965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773967e-425f-4f8c-902a-293c5aa8f383 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:02:41.725268 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773967e-425f-4f8c-902a-293c5aa8f383 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:41.726879 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773967e-425f-4f8c-902a-293c5aa8f383 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:41.741412 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1292/2578] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 08:02:41.745969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a37b0c-8cec-4a43-b3cd-7f426561f61d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:41.757681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a37b0c-8cec-4a43-b3cd-7f426561f61d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:41.764222 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c7524-cfe8-4c43-914d-fcdb27ad7c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:41.764649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c7524-cfe8-4c43-914d-fcdb27ad7c7d admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:41.764921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c7524-cfe8-4c43-914d-fcdb27ad7c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:41.765185 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c7524-cfe8-4c43-914d-fcdb27ad7c7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:41.765834 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c7524-cfe8-4c43-914d-fcdb27ad7c7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:41.767742 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c7524-cfe8-4c43-914d-fcdb27ad7c7d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:41.808144 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1287/2579] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:41 2022] POST /identity/v3/projects => generated 357 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:41.813652 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773967e-425f-4f8c-902a-293c5aa8f383 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:41.818797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c7524-cfe8-4c43-914d-fcdb27ad7c7d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:41.820710 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0d4c7524-cfe8-4c43-914d-fcdb27ad7c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:41.830323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773967e-425f-4f8c-902a-293c5aa8f383 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:41.869660 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:41.898353 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c7524-cfe8-4c43-914d-fcdb27ad7c7d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:41.904216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:41.906191 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:41.906503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:41.906786 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73 admin admin] PATH_INFO: `/v3/projects/a6b39e753f5847b295dbfd9169898345` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:41.908026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6b39e753f5847b295dbfd9169898345)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:41.909955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:41.920335 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b078d9c-656a-4961-a8a4-3311ab41069b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:41.921215 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b078d9c-656a-4961-a8a4-3311ab41069b admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:41.922261 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b078d9c-656a-4961-a8a4-3311ab41069b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:41.922781 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b078d9c-656a-4961-a8a4-3311ab41069b admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:41.923363 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b078d9c-656a-4961-a8a4-3311ab41069b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:41.925322 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b078d9c-656a-4961-a8a4-3311ab41069b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:41.953330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1293/2580] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:41 2022] POST /identity/v3/users => generated 279 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:41.954555 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b078d9c-656a-4961-a8a4-3311ab41069b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:41.965637 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b078d9c-656a-4961-a8a4-3311ab41069b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:41.967669 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6b39e753f5847b295dbfd9169898345', 'request_id': 'req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.974841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6b39e753f5847b295dbfd9169898345', 'request_id': 'req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.984191 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6b39e753f5847b295dbfd9169898345', 'request_id': 'req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:41.989581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:41.999580 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73 admin admin] The token cache is being invalidate because project a6b39e753f5847b295dbfd9169898345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:41.999882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:42.000557 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:42.002085 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1288/2581] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:41 2022] DELETE /identity/v3/projects/a6b39e753f5847b295dbfd9169898345 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:42.005804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:42.007208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:42.039162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b63154-1e9a-42de-befe-205aedfbe83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:42.040395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b63154-1e9a-42de-befe-205aedfbe83a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.040857 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b63154-1e9a-42de-befe-205aedfbe83a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.041285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b63154-1e9a-42de-befe-205aedfbe83a admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.042104 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b63154-1e9a-42de-befe-205aedfbe83a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:42.043733 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b63154-1e9a-42de-befe-205aedfbe83a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:42.060147 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1294/2582] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:42.063709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b63154-1e9a-42de-befe-205aedfbe83a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:42.066224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-96b63154-1e9a-42de-befe-205aedfbe83a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:42.079286 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b63154-1e9a-42de-befe-205aedfbe83a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:42.083197 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c5a1c-7f4f-4e94-a01e-2b6f4697db73 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:42.084628 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09527cc9-364f-4f41-921a-37a4cf2a94cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:42.085233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09527cc9-364f-4f41-921a-37a4cf2a94cd admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.085518 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09527cc9-364f-4f41-921a-37a4cf2a94cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.085810 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09527cc9-364f-4f41-921a-37a4cf2a94cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.086919 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09527cc9-364f-4f41-921a-37a4cf2a94cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:42.088538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09527cc9-364f-4f41-921a-37a4cf2a94cd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:42.088727 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08762aca-3234-4688-9228-a3eaf17f4d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:42.089922 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08762aca-3234-4688-9228-a3eaf17f4d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.089922 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08762aca-3234-4688-9228-a3eaf17f4d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.090126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08762aca-3234-4688-9228-a3eaf17f4d85 admin admin] PATH_INFO: `/v3/users/4698e65ceb664749a89260343bdbec0d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.091546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08762aca-3234-4688-9228-a3eaf17f4d85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4698e65ceb664749a89260343bdbec0d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:42.094062 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08762aca-3234-4688-9228-a3eaf17f4d85 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:42.099801 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1295/2583] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:42.104606 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09527cc9-364f-4f41-921a-37a4cf2a94cd admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:42.117304 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09527cc9-364f-4f41-921a-37a4cf2a94cd admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:42.122614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e0ac2-0a22-4ffa-826f-2261d1deb593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:42.123430 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e0ac2-0a22-4ffa-826f-2261d1deb593 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.124509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e0ac2-0a22-4ffa-826f-2261d1deb593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.125201 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e0ac2-0a22-4ffa-826f-2261d1deb593 admin admin] PATH_INFO: `/v3/projects/a0034c10c6404be19f19bb5c817c4807/users/e2c6d14a1af44afdaf8dafa7e69130d2/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.144524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e0ac2-0a22-4ffa-826f-2261d1deb593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0034c10c6404be19f19bb5c817c4807, user_id=e2c6d14a1af44afdaf8dafa7e69130d2, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:42.147310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e0ac2-0a22-4ffa-826f-2261d1deb593 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:42.158980 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-08762aca-3234-4688-9228-a3eaf17f4d85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4698e65ceb664749a89260343bdbec0d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:42.159672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-08762aca-3234-4688-9228-a3eaf17f4d85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4698e65ceb664749a89260343bdbec0d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:42.176412 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1296/2584] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:42 2022] PUT /identity/v3/projects/a0034c10c6404be19f19bb5c817c4807/users/e2c6d14a1af44afdaf8dafa7e69130d2/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:42.177670 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-08762aca-3234-4688-9228-a3eaf17f4d85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4698e65ceb664749a89260343bdbec0d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:42.178476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-08762aca-3234-4688-9228-a3eaf17f4d85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4698e65ceb664749a89260343bdbec0d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:42.178959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430aeb6a-0be3-4a04-bcc6-0b4e262c0275 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.179246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430aeb6a-0be3-4a04-bcc6-0b4e262c0275 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.179572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430aeb6a-0be3-4a04-bcc6-0b4e262c0275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.189277 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:42.194767 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-08762aca-3234-4688-9228-a3eaf17f4d85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4698e65ceb664749a89260343bdbec0d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:42.220698 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1289/2585] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:42 2022] DELETE /identity/v3/users/4698e65ceb664749a89260343bdbec0d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:42.227699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-430aeb6a-0be3-4a04-bcc6-0b4e262c0275 None None] MFA Rules not processed for user `e2c6d14a1af44afdaf8dafa7e69130d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:42.229455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-430aeb6a-0be3-4a04-bcc6-0b4e262c0275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:42.328226 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1297/2586] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:42 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:42.330490 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08762aca-3234-4688-9228-a3eaf17f4d85 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:42.332495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-08762aca-3234-4688-9228-a3eaf17f4d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:42.433026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08762aca-3234-4688-9228-a3eaf17f4d85 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:42.441752 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095391c-29fe-46fb-a95a-0f948d15e2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:42.442435 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095391c-29fe-46fb-a95a-0f948d15e2bd admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.442716 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095391c-29fe-46fb-a95a-0f948d15e2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.443318 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095391c-29fe-46fb-a95a-0f948d15e2bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.443642 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095391c-29fe-46fb-a95a-0f948d15e2bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:42.445624 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095391c-29fe-46fb-a95a-0f948d15e2bd admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:42.497801 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1290/2587] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:42 2022] POST /identity/v3/projects => generated 355 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:42.499308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430aeb6a-0be3-4a04-bcc6-0b4e262c0275 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:42.510295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430aeb6a-0be3-4a04-bcc6-0b4e262c0275 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:42.540555 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095391c-29fe-46fb-a95a-0f948d15e2bd admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:42.542437 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0095391c-29fe-46fb-a95a-0f948d15e2bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:42.579428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef7b7a-35c2-42a4-b499-6c82414b5dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:42.580072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef7b7a-35c2-42a4-b499-6c82414b5dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.580512 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef7b7a-35c2-42a4-b499-6c82414b5dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.581183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef7b7a-35c2-42a4-b499-6c82414b5dc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.581827 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef7b7a-35c2-42a4-b499-6c82414b5dc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:42.583804 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef7b7a-35c2-42a4-b499-6c82414b5dc0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:42.601856 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:42.608247 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1298/2588] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:42 2022] POST /identity/v3/users => generated 284 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:42.611417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef7b7a-35c2-42a4-b499-6c82414b5dc0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:42.620578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095391c-29fe-46fb-a95a-0f948d15e2bd admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:42.621115 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef7b7a-35c2-42a4-b499-6c82414b5dc0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:42.626152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b364581-a7a0-473f-806e-1ce125202a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:42.626334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f5e81-68bd-419a-8470-4069ffbc8ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:42.626770 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f5e81-68bd-419a-8470-4069ffbc8ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.626877 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b364581-a7a0-473f-806e-1ce125202a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.627049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f5e81-68bd-419a-8470-4069ffbc8ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.627261 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b364581-a7a0-473f-806e-1ce125202a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.627374 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f5e81-68bd-419a-8470-4069ffbc8ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.627608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b364581-a7a0-473f-806e-1ce125202a6a admin admin] PATH_INFO: `/v3/projects/b19e0992dac048438225ba6acaa9c873` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.627982 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f5e81-68bd-419a-8470-4069ffbc8ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:42.629042 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b364581-a7a0-473f-806e-1ce125202a6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b19e0992dac048438225ba6acaa9c873)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:42.629541 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f5e81-68bd-419a-8470-4069ffbc8ff8 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:42.631073 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b364581-a7a0-473f-806e-1ce125202a6a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:42.645337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1299/2589] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:42.646361 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f5e81-68bd-419a-8470-4069ffbc8ff8 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:42.657430 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f5e81-68bd-419a-8470-4069ffbc8ff8 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:42.662646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99190010-017f-421b-aa70-e4d302842038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:42.663358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99190010-017f-421b-aa70-e4d302842038 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.663640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99190010-017f-421b-aa70-e4d302842038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.664000 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99190010-017f-421b-aa70-e4d302842038 admin admin] PATH_INFO: `/v3/projects/7ab347c217cb464c9e77fe12ead1dd78/users/a1365d05c215408790289cba9ee83f52/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.681163 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99190010-017f-421b-aa70-e4d302842038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab347c217cb464c9e77fe12ead1dd78, user_id=a1365d05c215408790289cba9ee83f52, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:42.684994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99190010-017f-421b-aa70-e4d302842038 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:42.694396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5b364581-a7a0-473f-806e-1ce125202a6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b19e0992dac048438225ba6acaa9c873', 'request_id': 'req-5b364581-a7a0-473f-806e-1ce125202a6a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:42.701838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5b364581-a7a0-473f-806e-1ce125202a6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b19e0992dac048438225ba6acaa9c873', 'request_id': 'req-5b364581-a7a0-473f-806e-1ce125202a6a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:42.707718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1300/2590] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:42 2022] PUT /identity/v3/projects/7ab347c217cb464c9e77fe12ead1dd78/users/a1365d05c215408790289cba9ee83f52/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:42.711528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5b364581-a7a0-473f-806e-1ce125202a6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b19e0992dac048438225ba6acaa9c873', 'request_id': 'req-5b364581-a7a0-473f-806e-1ce125202a6a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:42.712719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99190010-017f-421b-aa70-e4d302842038 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:42.730422 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5b364581-a7a0-473f-806e-1ce125202a6a admin admin] The token cache is being invalidate because project b19e0992dac048438225ba6acaa9c873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:42.730772 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5b364581-a7a0-473f-806e-1ce125202a6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:42.731391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5b364581-a7a0-473f-806e-1ce125202a6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:42.736179 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1291/2591] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:42 2022] DELETE /identity/v3/projects/b19e0992dac048438225ba6acaa9c873 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:42.740684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b0a32-564e-45f9-ac71-a5321b23922b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.741133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b0a32-564e-45f9-ac71-a5321b23922b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.741549 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b0a32-564e-45f9-ac71-a5321b23922b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.750804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:42.770396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-698b0a32-564e-45f9-ac71-a5321b23922b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:42.771064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-698b0a32-564e-45f9-ac71-a5321b23922b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:42.792243 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99190010-017f-421b-aa70-e4d302842038 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:42.801680 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e8ca8-7026-4e9e-9d95-ed15ea1e2048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:42.802727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8ca8-7026-4e9e-9d95-ed15ea1e2048 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.803162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8ca8-7026-4e9e-9d95-ed15ea1e2048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.804429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8ca8-7026-4e9e-9d95-ed15ea1e2048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.805087 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e8ca8-7026-4e9e-9d95-ed15ea1e2048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:42.807546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e8ca8-7026-4e9e-9d95-ed15ea1e2048 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:42.821762 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1301/2592] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:42.833043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e8ca8-7026-4e9e-9d95-ed15ea1e2048 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:42.834802 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-870e8ca8-7026-4e9e-9d95-ed15ea1e2048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:42.850151 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e8ca8-7026-4e9e-9d95-ed15ea1e2048 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:42.852350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1292/2593] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:42.855017 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c59fa4-335c-4e5a-bf0a-6416fa1beebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:42.855705 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c59fa4-335c-4e5a-bf0a-6416fa1beebe admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.855997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c59fa4-335c-4e5a-bf0a-6416fa1beebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.856300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c59fa4-335c-4e5a-bf0a-6416fa1beebe admin admin] PATH_INFO: `/v3/projects/7ab347c217cb464c9e77fe12ead1dd78/users/a1365d05c215408790289cba9ee83f52/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.857213 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4447067-503b-406e-a2d4-69baec597225 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.857588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4447067-503b-406e-a2d4-69baec597225 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.857868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4447067-503b-406e-a2d4-69baec597225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.858275 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c59fa4-335c-4e5a-bf0a-6416fa1beebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab347c217cb464c9e77fe12ead1dd78, user_id=a1365d05c215408790289cba9ee83f52, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:42.861058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c59fa4-335c-4e5a-bf0a-6416fa1beebe admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:42.871955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1302/2594] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:42 2022] PUT /identity/v3/projects/7ab347c217cb464c9e77fe12ead1dd78/users/a1365d05c215408790289cba9ee83f52/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:42.878079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d4447067-503b-406e-a2d4-69baec597225 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:42.879293 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d4447067-503b-406e-a2d4-69baec597225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:42.884181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d87c4-eabe-4892-8913-632d25f50ada None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.884464 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d87c4-eabe-4892-8913-632d25f50ada None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.884735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d87c4-eabe-4892-8913-632d25f50ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.914041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-451d87c4-eabe-4892-8913-632d25f50ada None None] MFA Rules not processed for user `a1365d05c215408790289cba9ee83f52`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:42.915144 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-451d87c4-eabe-4892-8913-632d25f50ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:42.938889 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1293/2595] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:42.943412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4447067-503b-406e-a2d4-69baec597225 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:42.953182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4447067-503b-406e-a2d4-69baec597225 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:42.957870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498e182-57d0-4af0-b18c-2130a1d6acee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:42.958467 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498e182-57d0-4af0-b18c-2130a1d6acee admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.958743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498e182-57d0-4af0-b18c-2130a1d6acee admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.959015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498e182-57d0-4af0-b18c-2130a1d6acee admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.959653 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498e182-57d0-4af0-b18c-2130a1d6acee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:02:42.959959 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498e182-57d0-4af0-b18c-2130a1d6acee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:42.961585 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498e182-57d0-4af0-b18c-2130a1d6acee admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:42.972948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1294/2596] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:02:42.977128 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498e182-57d0-4af0-b18c-2130a1d6acee admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:42.986001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1303/2597] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:42 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:42.986859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498e182-57d0-4af0-b18c-2130a1d6acee admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:42.994173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24f930-5971-4550-ae51-9f7639310f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:42.994881 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24f930-5971-4550-ae51-9f7639310f26 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:42.995224 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24f930-5971-4550-ae51-9f7639310f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:42.995598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24f930-5971-4550-ae51-9f7639310f26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:42.996315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc24f930-5971-4550-ae51-9f7639310f26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:42.996833 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d87c4-eabe-4892-8913-632d25f50ada None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:42.998661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-451d87c4-eabe-4892-8913-632d25f50ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:42.998835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc24f930-5971-4550-ae51-9f7639310f26 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:43.028216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1295/2598] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:42 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:43.032415 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24f930-5971-4550-ae51-9f7639310f26 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:43.037486 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:43.064559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d87c4-eabe-4892-8913-632d25f50ada None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:43.101473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24f930-5971-4550-ae51-9f7639310f26 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:43.106150 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd17e2-b953-4494-92be-acf3c316de2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:43.106750 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd17e2-b953-4494-92be-acf3c316de2d admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:43.107031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd17e2-b953-4494-92be-acf3c316de2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:43.107828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd17e2-b953-4494-92be-acf3c316de2d admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:43.108640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd17e2-b953-4494-92be-acf3c316de2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:43.110639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd17e2-b953-4494-92be-acf3c316de2d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:43.122519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce69298-2d60-4cef-89af-774ad9e9822c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:43.123749 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce69298-2d60-4cef-89af-774ad9e9822c service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:43.123986 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce69298-2d60-4cef-89af-774ad9e9822c service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:43.124288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce69298-2d60-4cef-89af-774ad9e9822c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:43.127851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:43.137755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1296/2599] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:43 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:43.139245 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd17e2-b953-4494-92be-acf3c316de2d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:43.149684 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd17e2-b953-4494-92be-acf3c316de2d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:43.154535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcbd70-7479-40df-a685-a83cfcdd8a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:43.155295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcbd70-7479-40df-a685-a83cfcdd8a79 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:43.155570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcbd70-7479-40df-a685-a83cfcdd8a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:43.155841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcbd70-7479-40df-a685-a83cfcdd8a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:43.156502 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcbd70-7479-40df-a685-a83cfcdd8a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:43.158097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcbd70-7479-40df-a685-a83cfcdd8a79 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:43.168299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1297/2600] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:43.173230 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcbd70-7479-40df-a685-a83cfcdd8a79 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:43.182982 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcbd70-7479-40df-a685-a83cfcdd8a79 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:43.188562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57c34e-5757-4f93-a740-28346a11d743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:43.189144 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57c34e-5757-4f93-a740-28346a11d743 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:43.189418 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57c34e-5757-4f93-a740-28346a11d743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:43.189715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57c34e-5757-4f93-a740-28346a11d743 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:43.190334 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57c34e-5757-4f93-a740-28346a11d743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:43.191911 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57c34e-5757-4f93-a740-28346a11d743 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:43.193845 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce69298-2d60-4cef-89af-774ad9e9822c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:43.195260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce69298-2d60-4cef-89af-774ad9e9822c service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:43.202364 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1298/2601] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:43.205523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1304/2602] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:42 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:02:43.208001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57c34e-5757-4f93-a740-28346a11d743 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:43.218985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57c34e-5757-4f93-a740-28346a11d743 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:43.223765 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49486c6f-be1f-4e7e-b9af-13fd83d936ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:43.224552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49486c6f-be1f-4e7e-b9af-13fd83d936ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:43.224866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49486c6f-be1f-4e7e-b9af-13fd83d936ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:43.225247 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49486c6f-be1f-4e7e-b9af-13fd83d936ed admin admin] PATH_INFO: `/v3/projects/72983346d12c425e80e9e187f84a0990/users/87ea3a4618a34e588d5fd2159f6bc6c0/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:43.242790 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49486c6f-be1f-4e7e-b9af-13fd83d936ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=72983346d12c425e80e9e187f84a0990, user_id=87ea3a4618a34e588d5fd2159f6bc6c0, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:43.245479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49486c6f-be1f-4e7e-b9af-13fd83d936ed admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:43.260021 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1299/2603] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:43 2022] PUT /identity/v3/projects/72983346d12c425e80e9e187f84a0990/users/87ea3a4618a34e588d5fd2159f6bc6c0/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:43.284732 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce69298-2d60-4cef-89af-774ad9e9822c service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:43.294128 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce69298-2d60-4cef-89af-774ad9e9822c service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:43.325731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bbb4e-a741-462c-abd9-81ad8b228dc7 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:43.326770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bbb4e-a741-462c-abd9-81ad8b228dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:43.327236 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bbb4e-a741-462c-abd9-81ad8b228dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:43.347654 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3405650-24ad-431d-9efb-ab013d50f9cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:43.348291 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3405650-24ad-431d-9efb-ab013d50f9cd service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:43.348573 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3405650-24ad-431d-9efb-ab013d50f9cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:43.348845 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3405650-24ad-431d-9efb-ab013d50f9cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:43.350608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b3405650-24ad-431d-9efb-ab013d50f9cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:43.355719 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-be6bbb4e-a741-462c-abd9-81ad8b228dc7 None None] MFA Rules not processed for user `87ea3a4618a34e588d5fd2159f6bc6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:43.356916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-be6bbb4e-a741-462c-abd9-81ad8b228dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:43.406944 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:43.410311 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3405650-24ad-431d-9efb-ab013d50f9cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:43.412002 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3405650-24ad-431d-9efb-ab013d50f9cd service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:43.421802 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1305/2604] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:43 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:43.424985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1300/2605] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:43.429399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3405650-24ad-431d-9efb-ab013d50f9cd service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:43.496130 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3405650-24ad-431d-9efb-ab013d50f9cd service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:43.504369 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0649aa-61f5-4cb4-bcb7-a35639b8bb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:43.505163 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0649aa-61f5-4cb4-bcb7-a35639b8bb63 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:43.505437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0649aa-61f5-4cb4-bcb7-a35639b8bb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:43.505847 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0649aa-61f5-4cb4-bcb7-a35639b8bb63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:43.506488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0649aa-61f5-4cb4-bcb7-a35639b8bb63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:43.508727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0649aa-61f5-4cb4-bcb7-a35639b8bb63 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:43.546991 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1306/2606] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:43 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:43.547643 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bbb4e-a741-462c-abd9-81ad8b228dc7 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:43.561029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bbb4e-a741-462c-abd9-81ad8b228dc7 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:43.616348 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0649aa-61f5-4cb4-bcb7-a35639b8bb63 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:43.618081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7b0649aa-61f5-4cb4-bcb7-a35639b8bb63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:43.628511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9b230-4db2-4f95-9fbd-c5311cf66451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:43.629201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9b230-4db2-4f95-9fbd-c5311cf66451 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:43.629550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9b230-4db2-4f95-9fbd-c5311cf66451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:43.629829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9b230-4db2-4f95-9fbd-c5311cf66451 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:43.630464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9b230-4db2-4f95-9fbd-c5311cf66451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:43.632416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9b230-4db2-4f95-9fbd-c5311cf66451 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:43.654766 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1301/2607] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:43 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:43.658604 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9b230-4db2-4f95-9fbd-c5311cf66451 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:43.668436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9b230-4db2-4f95-9fbd-c5311cf66451 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:43.673169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85be30-d99a-4772-b8c0-d0fcc271b4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:43.673765 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85be30-d99a-4772-b8c0-d0fcc271b4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:43.674043 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85be30-d99a-4772-b8c0-d0fcc271b4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:43.674310 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85be30-d99a-4772-b8c0-d0fcc271b4e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:43.675651 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85be30-d99a-4772-b8c0-d0fcc271b4e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:43.677561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85be30-d99a-4772-b8c0-d0fcc271b4e5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:43.679847 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0649aa-61f5-4cb4-bcb7-a35639b8bb63 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:43.683553 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1139ac-c38a-4716-8622-392aa21c64a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:43.684199 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1139ac-c38a-4716-8622-392aa21c64a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:43.684484 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1139ac-c38a-4716-8622-392aa21c64a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:43.684758 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1139ac-c38a-4716-8622-392aa21c64a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:43.687546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2d1139ac-c38a-4716-8622-392aa21c64a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:43.688302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1302/2608] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:43.692073 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85be30-d99a-4772-b8c0-d0fcc271b4e5 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:43.700033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1139ac-c38a-4716-8622-392aa21c64a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:43.702401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1139ac-c38a-4716-8622-392aa21c64a3 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:43.702614 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85be30-d99a-4772-b8c0-d0fcc271b4e5 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:43.707221 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fda251-4472-47a2-a734-f61bb8d89c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:43.707933 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fda251-4472-47a2-a734-f61bb8d89c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:43.708252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fda251-4472-47a2-a734-f61bb8d89c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:43.708528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fda251-4472-47a2-a734-f61bb8d89c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:43.709177 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fda251-4472-47a2-a734-f61bb8d89c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:43.710746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fda251-4472-47a2-a734-f61bb8d89c7d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:43.712406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1307/2609] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:02:43.722246 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1303/2610] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:43.726177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1139ac-c38a-4716-8622-392aa21c64a3 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:43.736505 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1139ac-c38a-4716-8622-392aa21c64a3 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:43.741336 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227d40d-6413-44ab-8efc-8d01a7acbed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:43.741985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227d40d-6413-44ab-8efc-8d01a7acbed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:43.742293 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227d40d-6413-44ab-8efc-8d01a7acbed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:43.742568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227d40d-6413-44ab-8efc-8d01a7acbed4 admin admin] PATH_INFO: `/v3/projects/f3480e291d0a4346b320f2a22ef88d73/users/75eb6da42b1e4223b29e01c827489bf5/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:43.757768 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227d40d-6413-44ab-8efc-8d01a7acbed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3480e291d0a4346b320f2a22ef88d73, user_id=75eb6da42b1e4223b29e01c827489bf5, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:43.760532 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227d40d-6413-44ab-8efc-8d01a7acbed4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:43.773469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1308/2611] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:43 2022] PUT /identity/v3/projects/f3480e291d0a4346b320f2a22ef88d73/users/75eb6da42b1e4223b29e01c827489bf5/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:43.784836 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc9cca-47f0-405c-9d01-4714f36ec697 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:43.785121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc9cca-47f0-405c-9d01-4714f36ec697 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:43.785416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc9cca-47f0-405c-9d01-4714f36ec697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:43.807602 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:43.818090 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-28cc9cca-47f0-405c-9d01-4714f36ec697 None None] MFA Rules not processed for user `75eb6da42b1e4223b29e01c827489bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:43.819306 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-28cc9cca-47f0-405c-9d01-4714f36ec697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:43.894942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1304/2612] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:43.899336 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227d40d-6413-44ab-8efc-8d01a7acbed4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:43.968008 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:43.974306 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227d40d-6413-44ab-8efc-8d01a7acbed4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:43.981357 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3994c-2443-49ca-ba73-c2156441227b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:43.982103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3994c-2443-49ca-ba73-c2156441227b admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:43.982499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3994c-2443-49ca-ba73-c2156441227b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:43.982976 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3994c-2443-49ca-ba73-c2156441227b admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:43.983947 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c3994c-2443-49ca-ba73-c2156441227b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:43.986364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c3994c-2443-49ca-ba73-c2156441227b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:44.017421 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1309/2613] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:43 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:44.022580 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc9cca-47f0-405c-9d01-4714f36ec697 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:44.036009 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc9cca-47f0-405c-9d01-4714f36ec697 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:44.106422 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026451d-39ff-4ad6-8b13-b3d27be522c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:44.108234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026451d-39ff-4ad6-8b13-b3d27be522c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:44.108419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026451d-39ff-4ad6-8b13-b3d27be522c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:44.108693 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026451d-39ff-4ad6-8b13-b3d27be522c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:44.109325 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026451d-39ff-4ad6-8b13-b3d27be522c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:44.111363 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026451d-39ff-4ad6-8b13-b3d27be522c2 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:44.138487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1305/2614] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:44 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:44.142275 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3994c-2443-49ca-ba73-c2156441227b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:44.153667 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3994c-2443-49ca-ba73-c2156441227b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:44.159584 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69993e-515f-4a4e-b8bb-0da80909aa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:44.160283 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69993e-515f-4a4e-b8bb-0da80909aa7b admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:44.160632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69993e-515f-4a4e-b8bb-0da80909aa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:44.160974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69993e-515f-4a4e-b8bb-0da80909aa7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:44.161740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69993e-515f-4a4e-b8bb-0da80909aa7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:44.163496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69993e-515f-4a4e-b8bb-0da80909aa7b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:44.176264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1310/2615] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:44.180488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026451d-39ff-4ad6-8b13-b3d27be522c2 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:44.195120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026451d-39ff-4ad6-8b13-b3d27be522c2 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:44.199736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1cec4-8dde-4cf4-a736-6973f0f2fe71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:44.200528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1cec4-8dde-4cf4-a736-6973f0f2fe71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:44.200868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1cec4-8dde-4cf4-a736-6973f0f2fe71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:44.201316 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1cec4-8dde-4cf4-a736-6973f0f2fe71 admin admin] PATH_INFO: `/v3/projects/21e26ba936834316b7a6d5e3f1796509/users/98732edd40494bbda47ae05b86dd2515/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:44.218135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1cec4-8dde-4cf4-a736-6973f0f2fe71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e26ba936834316b7a6d5e3f1796509, user_id=98732edd40494bbda47ae05b86dd2515, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:44.220934 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1cec4-8dde-4cf4-a736-6973f0f2fe71 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:44.237806 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1306/2616] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:44 2022] PUT /identity/v3/projects/21e26ba936834316b7a6d5e3f1796509/users/98732edd40494bbda47ae05b86dd2515/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:44.242009 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69993e-515f-4a4e-b8bb-0da80909aa7b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:44.257972 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69993e-515f-4a4e-b8bb-0da80909aa7b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:44.323702 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074b9f0-e6b3-43b7-a0e6-1aa770dc583a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:44.324414 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074b9f0-e6b3-43b7-a0e6-1aa770dc583a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:44.324761 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074b9f0-e6b3-43b7-a0e6-1aa770dc583a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:44.325097 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074b9f0-e6b3-43b7-a0e6-1aa770dc583a admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:44.325804 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074b9f0-e6b3-43b7-a0e6-1aa770dc583a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:44.327542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074b9f0-e6b3-43b7-a0e6-1aa770dc583a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:44.341115 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1311/2617] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:44.342949 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1cec4-8dde-4cf4-a736-6973f0f2fe71 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:44.353810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1cec4-8dde-4cf4-a736-6973f0f2fe71 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:44.359457 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7e2e7-7a1e-45b8-8ab4-29142d947e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:44.360146 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7e2e7-7a1e-45b8-8ab4-29142d947e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:44.360431 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7e2e7-7a1e-45b8-8ab4-29142d947e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:44.360707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7e2e7-7a1e-45b8-8ab4-29142d947e1a admin admin] PATH_INFO: `/v3/projects/21e26ba936834316b7a6d5e3f1796509/users/98732edd40494bbda47ae05b86dd2515/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:44.362958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a7e2e7-7a1e-45b8-8ab4-29142d947e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e26ba936834316b7a6d5e3f1796509, user_id=98732edd40494bbda47ae05b86dd2515, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:44.365334 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a7e2e7-7a1e-45b8-8ab4-29142d947e1a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:44.376308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1307/2618] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:44 2022] PUT /identity/v3/projects/21e26ba936834316b7a6d5e3f1796509/users/98732edd40494bbda47ae05b86dd2515/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:44.387680 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76947f7-e909-4021-9ec5-db0928b03d42 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:44.388005 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76947f7-e909-4021-9ec5-db0928b03d42 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:44.388318 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76947f7-e909-4021-9ec5-db0928b03d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:44.421170 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c76947f7-e909-4021-9ec5-db0928b03d42 None None] MFA Rules not processed for user `98732edd40494bbda47ae05b86dd2515`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:44.422399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c76947f7-e909-4021-9ec5-db0928b03d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:44.491576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1312/2619] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:44.505327 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7e2e7-7a1e-45b8-8ab4-29142d947e1a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:44.566155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7e2e7-7a1e-45b8-8ab4-29142d947e1a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:44.569841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513fa5c5-cfca-4dbe-ae01-6ce7c28667e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:44.570442 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513fa5c5-cfca-4dbe-ae01-6ce7c28667e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:44.570740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513fa5c5-cfca-4dbe-ae01-6ce7c28667e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:44.570993 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513fa5c5-cfca-4dbe-ae01-6ce7c28667e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:44.575281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:44.637556 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513fa5c5-cfca-4dbe-ae01-6ce7c28667e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:44.639830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513fa5c5-cfca-4dbe-ae01-6ce7c28667e1 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:44.659729 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1308/2620] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:46.516762 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76947f7-e909-4021-9ec5-db0928b03d42 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:46.530958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76947f7-e909-4021-9ec5-db0928b03d42 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:46.588966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe368cf-3f62-4b09-a163-ad28294ab191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:46.589602 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe368cf-3f62-4b09-a163-ad28294ab191 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:46.589881 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe368cf-3f62-4b09-a163-ad28294ab191 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:46.590151 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe368cf-3f62-4b09-a163-ad28294ab191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:46.718887 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:46.740971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe368cf-3f62-4b09-a163-ad28294ab191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:46.742823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe368cf-3f62-4b09-a163-ad28294ab191 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:46.767120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1313/2621] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:47.398034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513fa5c5-cfca-4dbe-ae01-6ce7c28667e1 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:47.409049 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513fa5c5-cfca-4dbe-ae01-6ce7c28667e1 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:47.421790 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbcde3-5e05-4552-8564-b8b31e1835ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:47.422433 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbcde3-5e05-4552-8564-b8b31e1835ee service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:47.422712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbcde3-5e05-4552-8564-b8b31e1835ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:47.422983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbcde3-5e05-4552-8564-b8b31e1835ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:47.433614 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbcde3-5e05-4552-8564-b8b31e1835ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:47.435920 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbcde3-5e05-4552-8564-b8b31e1835ee service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:47.452222 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1309/2622] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:47 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:47.757941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe368cf-3f62-4b09-a163-ad28294ab191 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:47.759553 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bfe368cf-3f62-4b09-a163-ad28294ab191 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:47.863219 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe368cf-3f62-4b09-a163-ad28294ab191 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:47.868966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e229bc-ba50-41ad-a2e4-add809c5bc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:47.870355 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e229bc-ba50-41ad-a2e4-add809c5bc5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:47.870639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e229bc-ba50-41ad-a2e4-add809c5bc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:47.870917 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e229bc-ba50-41ad-a2e4-add809c5bc5f admin admin] PATH_INFO: `/v3/users/55b97e954da94f909f8f5ffff515509e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:47.872961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e229bc-ba50-41ad-a2e4-add809c5bc5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=55b97e954da94f909f8f5ffff515509e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:47.895330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e229bc-ba50-41ad-a2e4-add809c5bc5f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:47.955753 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a2e229bc-ba50-41ad-a2e4-add809c5bc5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55b97e954da94f909f8f5ffff515509e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:47.969158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a2e229bc-ba50-41ad-a2e4-add809c5bc5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55b97e954da94f909f8f5ffff515509e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:47.977352 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a2e229bc-ba50-41ad-a2e4-add809c5bc5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55b97e954da94f909f8f5ffff515509e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:47.977979 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a2e229bc-ba50-41ad-a2e4-add809c5bc5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55b97e954da94f909f8f5ffff515509e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:47.986610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a2e229bc-ba50-41ad-a2e4-add809c5bc5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55b97e954da94f909f8f5ffff515509e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:47.992845 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1314/2623] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:47 2022] DELETE /identity/v3/users/55b97e954da94f909f8f5ffff515509e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:48.190300 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbcde3-5e05-4552-8564-b8b31e1835ee service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:48.202296 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbcde3-5e05-4552-8564-b8b31e1835ee service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:48.265617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d80c55-2397-42b5-9624-ff29d10d46ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:48.266270 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d80c55-2397-42b5-9624-ff29d10d46ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:48.266548 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d80c55-2397-42b5-9624-ff29d10d46ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:48.266818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d80c55-2397-42b5-9624-ff29d10d46ee admin admin] PATH_INFO: `/v3/projects/3264396aada44ba89f39cfe0776e183e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:48.267919 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d80c55-2397-42b5-9624-ff29d10d46ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=3264396aada44ba89f39cfe0776e183e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:48.269837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d80c55-2397-42b5-9624-ff29d10d46ee admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:48.317224 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-32d80c55-2397-42b5-9624-ff29d10d46ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3264396aada44ba89f39cfe0776e183e', 'request_id': 'req-32d80c55-2397-42b5-9624-ff29d10d46ee'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:48.322708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-32d80c55-2397-42b5-9624-ff29d10d46ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3264396aada44ba89f39cfe0776e183e', 'request_id': 'req-32d80c55-2397-42b5-9624-ff29d10d46ee'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:48.329650 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-32d80c55-2397-42b5-9624-ff29d10d46ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3264396aada44ba89f39cfe0776e183e', 'request_id': 'req-32d80c55-2397-42b5-9624-ff29d10d46ee'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:48.344093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-32d80c55-2397-42b5-9624-ff29d10d46ee admin admin] The token cache is being invalidate because project 3264396aada44ba89f39cfe0776e183e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:48.344398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-32d80c55-2397-42b5-9624-ff29d10d46ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:48.345068 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-32d80c55-2397-42b5-9624-ff29d10d46ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:48.346853 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1310/2624] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:48 2022] DELETE /identity/v3/projects/3264396aada44ba89f39cfe0776e183e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:48.351676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbc967-66cc-47c2-a4a0-931405076c71 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:48.351937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbc967-66cc-47c2-a4a0-931405076c71 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:48.352248 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbc967-66cc-47c2-a4a0-931405076c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:48.372015 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9fdbc967-66cc-47c2-a4a0-931405076c71 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:48.373366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9fdbc967-66cc-47c2-a4a0-931405076c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:48.434123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1315/2625] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:48.442576 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3b7f3-f54f-4fb4-9afb-f01053f1908b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:48.443096 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3b7f3-f54f-4fb4-9afb-f01053f1908b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:48.443465 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3b7f3-f54f-4fb4-9afb-f01053f1908b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:48.464688 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-87e3b7f3-f54f-4fb4-9afb-f01053f1908b None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:48.465953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-87e3b7f3-f54f-4fb4-9afb-f01053f1908b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:48.475826 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1311/2626] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:48.480064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbc967-66cc-47c2-a4a0-931405076c71 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:48.490422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbc967-66cc-47c2-a4a0-931405076c71 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:48.496399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372c8f9-d49d-415a-a2cd-7438bbfdbcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:48.497381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372c8f9-d49d-415a-a2cd-7438bbfdbcb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:48.497769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372c8f9-d49d-415a-a2cd-7438bbfdbcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:48.498126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372c8f9-d49d-415a-a2cd-7438bbfdbcb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:48.498993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372c8f9-d49d-415a-a2cd-7438bbfdbcb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:02:48.499395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372c8f9-d49d-415a-a2cd-7438bbfdbcb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:48.501908 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372c8f9-d49d-415a-a2cd-7438bbfdbcb0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:48.510108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:48.522068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1316/2627] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:02:48.523194 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3b7f3-f54f-4fb4-9afb-f01053f1908b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:48.534028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3b7f3-f54f-4fb4-9afb-f01053f1908b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:48.540246 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3023d-9adb-430e-9e1f-4d22b810a322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:48.540859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3023d-9adb-430e-9e1f-4d22b810a322 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:48.541135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3023d-9adb-430e-9e1f-4d22b810a322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:48.541407 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3023d-9adb-430e-9e1f-4d22b810a322 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:48.542046 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca3023d-9adb-430e-9e1f-4d22b810a322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:48.544645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca3023d-9adb-430e-9e1f-4d22b810a322 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:48.562192 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:48.576783 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1312/2628] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:48 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:48.580949 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372c8f9-d49d-415a-a2cd-7438bbfdbcb0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:48.591012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372c8f9-d49d-415a-a2cd-7438bbfdbcb0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:48.655304 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc739d00-89d5-4fa5-839c-e7ea61d44da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:48.655904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc739d00-89d5-4fa5-839c-e7ea61d44da2 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:48.656341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc739d00-89d5-4fa5-839c-e7ea61d44da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:48.656619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc739d00-89d5-4fa5-839c-e7ea61d44da2 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:48.657365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc739d00-89d5-4fa5-839c-e7ea61d44da2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:48.659128 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc739d00-89d5-4fa5-839c-e7ea61d44da2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:48.686530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1317/2629] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:48 2022] POST /identity/v3/users => generated 303 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:48.688103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3023d-9adb-430e-9e1f-4d22b810a322 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:48.699348 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3023d-9adb-430e-9e1f-4d22b810a322 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:48.705129 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37568f03-2175-459e-92cd-0b0c10d6b6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:48.705803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37568f03-2175-459e-92cd-0b0c10d6b6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:48.706152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37568f03-2175-459e-92cd-0b0c10d6b6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:48.706495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37568f03-2175-459e-92cd-0b0c10d6b6b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:48.707202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37568f03-2175-459e-92cd-0b0c10d6b6b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:48.708941 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37568f03-2175-459e-92cd-0b0c10d6b6b0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:48.723128 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1313/2630] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:48.724321 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc739d00-89d5-4fa5-839c-e7ea61d44da2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:48.735063 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc739d00-89d5-4fa5-839c-e7ea61d44da2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:48.739655 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc4e02-a982-4135-90af-db9ba290bc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:48.740285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc4e02-a982-4135-90af-db9ba290bc56 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:48.740561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc4e02-a982-4135-90af-db9ba290bc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:48.740831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc4e02-a982-4135-90af-db9ba290bc56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:48.741483 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc4e02-a982-4135-90af-db9ba290bc56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:48.743077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc4e02-a982-4135-90af-db9ba290bc56 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:48.756179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1318/2631] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:48.760820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37568f03-2175-459e-92cd-0b0c10d6b6b0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:48.770928 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37568f03-2175-459e-92cd-0b0c10d6b6b0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:48.775537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327cd0d-6c6d-4c29-a6ed-579e5dd18aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:48.776255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327cd0d-6c6d-4c29-a6ed-579e5dd18aee admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:48.776528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327cd0d-6c6d-4c29-a6ed-579e5dd18aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:48.776798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327cd0d-6c6d-4c29-a6ed-579e5dd18aee admin admin] PATH_INFO: `/v3/projects/83e0acc4f39144a2b4fff125237586a4/users/2da594ef36224a9184f73bf7dfbfee56/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:48.791959 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327cd0d-6c6d-4c29-a6ed-579e5dd18aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=83e0acc4f39144a2b4fff125237586a4, user_id=2da594ef36224a9184f73bf7dfbfee56, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:48.794918 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327cd0d-6c6d-4c29-a6ed-579e5dd18aee admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:48.809515 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1314/2632] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:48 2022] PUT /identity/v3/projects/83e0acc4f39144a2b4fff125237586a4/users/2da594ef36224a9184f73bf7dfbfee56/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:48.822048 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0230a68-b882-4cd0-9be6-0ebed70aa369 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:48.822524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0230a68-b882-4cd0-9be6-0ebed70aa369 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:48.822647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0230a68-b882-4cd0-9be6-0ebed70aa369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:48.855747 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f0230a68-b882-4cd0-9be6-0ebed70aa369 None None] MFA Rules not processed for user `2da594ef36224a9184f73bf7dfbfee56`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:48.857020 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f0230a68-b882-4cd0-9be6-0ebed70aa369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:48.937984 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1319/2633] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:48 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:48.942632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327cd0d-6c6d-4c29-a6ed-579e5dd18aee admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:49.008881 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327cd0d-6c6d-4c29-a6ed-579e5dd18aee admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:49.014119 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984387f7-3866-4d09-a68e-36a48e121f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:49.014777 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984387f7-3866-4d09-a68e-36a48e121f42 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:49.015070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984387f7-3866-4d09-a68e-36a48e121f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:49.021792 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984387f7-3866-4d09-a68e-36a48e121f42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:49.022482 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984387f7-3866-4d09-a68e-36a48e121f42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:49.024562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984387f7-3866-4d09-a68e-36a48e121f42 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:49.057420 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1315/2634] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:48 2022] POST /identity/v3/projects => generated 403 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:49.059027 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0230a68-b882-4cd0-9be6-0ebed70aa369 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:49.071775 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0230a68-b882-4cd0-9be6-0ebed70aa369 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:49.130794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d7080-3e5a-494c-9632-caf603bba18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:49.131535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d7080-3e5a-494c-9632-caf603bba18d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:49.131815 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d7080-3e5a-494c-9632-caf603bba18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:49.132100 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d7080-3e5a-494c-9632-caf603bba18d admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:49.132758 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d7080-3e5a-494c-9632-caf603bba18d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:49.134744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d7080-3e5a-494c-9632-caf603bba18d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:49.147150 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:49.176429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1320/2635] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:49 2022] POST /identity/v3/users => generated 308 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:49.180452 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984387f7-3866-4d09-a68e-36a48e121f42 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:49.191956 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984387f7-3866-4d09-a68e-36a48e121f42 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:49.196743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df9a34-5021-498e-86c5-4cc69065faaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:49.197372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df9a34-5021-498e-86c5-4cc69065faaa admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:49.197648 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df9a34-5021-498e-86c5-4cc69065faaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:49.197940 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df9a34-5021-498e-86c5-4cc69065faaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:49.198571 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df9a34-5021-498e-86c5-4cc69065faaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:49.200178 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df9a34-5021-498e-86c5-4cc69065faaa admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:49.211822 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1316/2636] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:49.216327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d7080-3e5a-494c-9632-caf603bba18d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:49.229915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d7080-3e5a-494c-9632-caf603bba18d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:49.237570 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e8769-f79a-445b-a482-23273af87e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:49.238574 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e8769-f79a-445b-a482-23273af87e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:49.239069 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e8769-f79a-445b-a482-23273af87e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:49.239677 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e8769-f79a-445b-a482-23273af87e6a admin admin] PATH_INFO: `/v3/projects/4ddb2c4ff8c94c7db4a4149f1f71fcf6/users/52e5e32754fe4fd1b4cb8e23fc03d911/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:49.265310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e8769-f79a-445b-a482-23273af87e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ddb2c4ff8c94c7db4a4149f1f71fcf6, user_id=52e5e32754fe4fd1b4cb8e23fc03d911, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:49.266772 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e8769-f79a-445b-a482-23273af87e6a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:49.281440 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1321/2637] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:49 2022] PUT /identity/v3/projects/4ddb2c4ff8c94c7db4a4149f1f71fcf6/users/52e5e32754fe4fd1b4cb8e23fc03d911/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:49.287711 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df9a34-5021-498e-86c5-4cc69065faaa admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:49.298254 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df9a34-5021-498e-86c5-4cc69065faaa admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:49.374027 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c067b-a178-4129-bcdf-06b8f0309dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:49.374893 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c067b-a178-4129-bcdf-06b8f0309dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:49.375344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c067b-a178-4129-bcdf-06b8f0309dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:49.375612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c067b-a178-4129-bcdf-06b8f0309dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:49.376288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c067b-a178-4129-bcdf-06b8f0309dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:49.378041 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c067b-a178-4129-bcdf-06b8f0309dc9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:49.396113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1317/2638] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:49.399664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e8769-f79a-445b-a482-23273af87e6a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:49.412538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e8769-f79a-445b-a482-23273af87e6a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:49.424590 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b65cc-2f93-4950-9dc9-878c39693fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:49.425500 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b65cc-2f93-4950-9dc9-878c39693fa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:49.425855 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b65cc-2f93-4950-9dc9-878c39693fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:49.426202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b65cc-2f93-4950-9dc9-878c39693fa7 admin admin] PATH_INFO: `/v3/projects/4ddb2c4ff8c94c7db4a4149f1f71fcf6/users/52e5e32754fe4fd1b4cb8e23fc03d911/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:49.428354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b65cc-2f93-4950-9dc9-878c39693fa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ddb2c4ff8c94c7db4a4149f1f71fcf6, user_id=52e5e32754fe4fd1b4cb8e23fc03d911, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:49.431223 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b65cc-2f93-4950-9dc9-878c39693fa7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:49.444032 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1322/2639] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:49 2022] PUT /identity/v3/projects/4ddb2c4ff8c94c7db4a4149f1f71fcf6/users/52e5e32754fe4fd1b4cb8e23fc03d911/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:49.456824 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62093657-1cac-4c8e-b9d1-68fe74d374d5 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:49.457366 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62093657-1cac-4c8e-b9d1-68fe74d374d5 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:49.457366 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62093657-1cac-4c8e-b9d1-68fe74d374d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:49.488676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-62093657-1cac-4c8e-b9d1-68fe74d374d5 None None] MFA Rules not processed for user `52e5e32754fe4fd1b4cb8e23fc03d911`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:49.491336 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-62093657-1cac-4c8e-b9d1-68fe74d374d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:49.509786 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:49.565513 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1318/2640] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:49 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:49.576887 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b65cc-2f93-4950-9dc9-878c39693fa7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:49.578608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-587b65cc-2f93-4950-9dc9-878c39693fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:49.634498 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b65cc-2f93-4950-9dc9-878c39693fa7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:49.638084 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7d5ce-b5ec-4abb-abe2-a102d36bce09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:49.639403 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7d5ce-b5ec-4abb-abe2-a102d36bce09 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:49.639674 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7d5ce-b5ec-4abb-abe2-a102d36bce09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:49.639933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7d5ce-b5ec-4abb-abe2-a102d36bce09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:49.649340 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7d5ce-b5ec-4abb-abe2-a102d36bce09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:49.651002 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7d5ce-b5ec-4abb-abe2-a102d36bce09 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:49.660720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1323/2641] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:49 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:49.818503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62093657-1cac-4c8e-b9d1-68fe74d374d5 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:49.887204 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62093657-1cac-4c8e-b9d1-68fe74d374d5 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:49.891811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c1ead-8e86-4001-ac05-3c680c7b6dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:49.892462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c1ead-8e86-4001-ac05-3c680c7b6dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:49.892737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c1ead-8e86-4001-ac05-3c680c7b6dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:49.893010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c1ead-8e86-4001-ac05-3c680c7b6dfc admin admin] PATH_INFO: `/v3/users/2da594ef36224a9184f73bf7dfbfee56` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:49.894132 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c1ead-8e86-4001-ac05-3c680c7b6dfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2da594ef36224a9184f73bf7dfbfee56)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:49.896052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c1ead-8e86-4001-ac05-3c680c7b6dfc admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:49.957141 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c52c1ead-8e86-4001-ac05-3c680c7b6dfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2da594ef36224a9184f73bf7dfbfee56'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:49.958004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c52c1ead-8e86-4001-ac05-3c680c7b6dfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2da594ef36224a9184f73bf7dfbfee56'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:49.976100 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c52c1ead-8e86-4001-ac05-3c680c7b6dfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2da594ef36224a9184f73bf7dfbfee56'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:49.977010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c52c1ead-8e86-4001-ac05-3c680c7b6dfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2da594ef36224a9184f73bf7dfbfee56'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:49.986291 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c52c1ead-8e86-4001-ac05-3c680c7b6dfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2da594ef36224a9184f73bf7dfbfee56'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:50.007725 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1319/2642] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:49 2022] DELETE /identity/v3/users/2da594ef36224a9184f73bf7dfbfee56 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:50.009513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6ab3e-9d80-46b8-b045-d1936741478d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:50.009977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6ab3e-9d80-46b8-b045-d1936741478d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:50.009977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6ab3e-9d80-46b8-b045-d1936741478d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:50.049567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-68c6ab3e-9d80-46b8-b045-d1936741478d None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:50.050797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-68c6ab3e-9d80-46b8-b045-d1936741478d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:50.120064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1324/2643] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:50.130723 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c1ead-8e86-4001-ac05-3c680c7b6dfc admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:50.132180 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c52c1ead-8e86-4001-ac05-3c680c7b6dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:50.171601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:50.197073 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c1ead-8e86-4001-ac05-3c680c7b6dfc admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:50.201101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def874d-b247-422c-a872-63a39e779401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:50.201737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def874d-b247-422c-a872-63a39e779401 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:50.202014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def874d-b247-422c-a872-63a39e779401 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:50.202290 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def874d-b247-422c-a872-63a39e779401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:50.211371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def874d-b247-422c-a872-63a39e779401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:50.213020 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def874d-b247-422c-a872-63a39e779401 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:50.223600 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1320/2644] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:02:50.516829 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6ab3e-9d80-46b8-b045-d1936741478d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:50.518359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-68c6ab3e-9d80-46b8-b045-d1936741478d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:50.545896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6ab3e-9d80-46b8-b045-d1936741478d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:50.550977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a05528-1317-4ca2-9fa9-0aeacdd68f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:50.551610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a05528-1317-4ca2-9fa9-0aeacdd68f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:50.551885 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a05528-1317-4ca2-9fa9-0aeacdd68f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:50.552193 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a05528-1317-4ca2-9fa9-0aeacdd68f38 admin admin] PATH_INFO: `/v3/projects/83e0acc4f39144a2b4fff125237586a4` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:50.553344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a05528-1317-4ca2-9fa9-0aeacdd68f38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83e0acc4f39144a2b4fff125237586a4)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:50.555222 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a05528-1317-4ca2-9fa9-0aeacdd68f38 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:50.604745 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-64a05528-1317-4ca2-9fa9-0aeacdd68f38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83e0acc4f39144a2b4fff125237586a4', 'request_id': 'req-64a05528-1317-4ca2-9fa9-0aeacdd68f38'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:50.613924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:50.615080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-64a05528-1317-4ca2-9fa9-0aeacdd68f38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83e0acc4f39144a2b4fff125237586a4', 'request_id': 'req-64a05528-1317-4ca2-9fa9-0aeacdd68f38'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:50.625283 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-64a05528-1317-4ca2-9fa9-0aeacdd68f38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83e0acc4f39144a2b4fff125237586a4', 'request_id': 'req-64a05528-1317-4ca2-9fa9-0aeacdd68f38'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:50.639868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-64a05528-1317-4ca2-9fa9-0aeacdd68f38 admin admin] The token cache is being invalidate because project 83e0acc4f39144a2b4fff125237586a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:50.640271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-64a05528-1317-4ca2-9fa9-0aeacdd68f38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:50.640979 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-64a05528-1317-4ca2-9fa9-0aeacdd68f38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:50.643113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1325/2645] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:50 2022] DELETE /identity/v3/projects/83e0acc4f39144a2b4fff125237586a4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:50.647274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def874d-b247-422c-a872-63a39e779401 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:50.657277 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def874d-b247-422c-a872-63a39e779401 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:50.726533 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d349eae-9ace-4a22-ad68-5ccc2a696a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:50.727165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d349eae-9ace-4a22-ad68-5ccc2a696a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:50.727440 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d349eae-9ace-4a22-ad68-5ccc2a696a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:50.727709 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d349eae-9ace-4a22-ad68-5ccc2a696a76 admin admin] PATH_INFO: `/v3/users/52e5e32754fe4fd1b4cb8e23fc03d911` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:50.728844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d349eae-9ace-4a22-ad68-5ccc2a696a76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52e5e32754fe4fd1b4cb8e23fc03d911)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:50.730746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d349eae-9ace-4a22-ad68-5ccc2a696a76 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:50.778440 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3d349eae-9ace-4a22-ad68-5ccc2a696a76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52e5e32754fe4fd1b4cb8e23fc03d911'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:50.779113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3d349eae-9ace-4a22-ad68-5ccc2a696a76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52e5e32754fe4fd1b4cb8e23fc03d911'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:50.791017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3d349eae-9ace-4a22-ad68-5ccc2a696a76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52e5e32754fe4fd1b4cb8e23fc03d911'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:50.791622 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3d349eae-9ace-4a22-ad68-5ccc2a696a76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52e5e32754fe4fd1b4cb8e23fc03d911'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:50.800321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3d349eae-9ace-4a22-ad68-5ccc2a696a76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52e5e32754fe4fd1b4cb8e23fc03d911'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:50.815422 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1321/2646] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:50 2022] DELETE /identity/v3/users/52e5e32754fe4fd1b4cb8e23fc03d911 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:51.105933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a05528-1317-4ca2-9fa9-0aeacdd68f38 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:51.107497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-64a05528-1317-4ca2-9fa9-0aeacdd68f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:51.121108 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a05528-1317-4ca2-9fa9-0aeacdd68f38 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:51.182591 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c011e23-7254-4c52-a084-248404e65986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:51.183257 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c011e23-7254-4c52-a084-248404e65986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:51.183543 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c011e23-7254-4c52-a084-248404e65986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:51.183820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c011e23-7254-4c52-a084-248404e65986 admin admin] PATH_INFO: `/v3/projects/4ddb2c4ff8c94c7db4a4149f1f71fcf6` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:51.184988 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c011e23-7254-4c52-a084-248404e65986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ddb2c4ff8c94c7db4a4149f1f71fcf6)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:51.187865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c011e23-7254-4c52-a084-248404e65986 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:51.239728 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6c011e23-7254-4c52-a084-248404e65986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ddb2c4ff8c94c7db4a4149f1f71fcf6', 'request_id': 'req-6c011e23-7254-4c52-a084-248404e65986'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:51.245551 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6c011e23-7254-4c52-a084-248404e65986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ddb2c4ff8c94c7db4a4149f1f71fcf6', 'request_id': 'req-6c011e23-7254-4c52-a084-248404e65986'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:51.253132 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6c011e23-7254-4c52-a084-248404e65986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ddb2c4ff8c94c7db4a4149f1f71fcf6', 'request_id': 'req-6c011e23-7254-4c52-a084-248404e65986'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:51.266721 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6c011e23-7254-4c52-a084-248404e65986 admin admin] The token cache is being invalidate because project 4ddb2c4ff8c94c7db4a4149f1f71fcf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:51.266998 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6c011e23-7254-4c52-a084-248404e65986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:51.267630 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6c011e23-7254-4c52-a084-248404e65986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:51.269152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1326/2647] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:51 2022] DELETE /identity/v3/projects/4ddb2c4ff8c94c7db4a4149f1f71fcf6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:51.274204 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd143d-70d6-49ca-9db7-ec7290a97dfc None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:51.274491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd143d-70d6-49ca-9db7-ec7290a97dfc None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:51.274764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd143d-70d6-49ca-9db7-ec7290a97dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:51.294191 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9bdd143d-70d6-49ca-9db7-ec7290a97dfc None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:51.295443 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9bdd143d-70d6-49ca-9db7-ec7290a97dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:51.343770 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:51.357660 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1322/2648] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:51.365285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4984b-0672-4549-bcaa-3124d40a8d9d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:51.365565 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4984b-0672-4549-bcaa-3124d40a8d9d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:51.365831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4984b-0672-4549-bcaa-3124d40a8d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:51.393043 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7ab4984b-0672-4549-bcaa-3124d40a8d9d None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:51.394275 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7ab4984b-0672-4549-bcaa-3124d40a8d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:51.404750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1327/2649] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:51.408921 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd143d-70d6-49ca-9db7-ec7290a97dfc None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:51.418623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4984b-0672-4549-bcaa-3124d40a8d9d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:51.420610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7ab4984b-0672-4549-bcaa-3124d40a8d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:51.422593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd143d-70d6-49ca-9db7-ec7290a97dfc None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:51.427239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd04e1-9ff3-4f1a-b4ff-b43c7da8f7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:51.427842 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd04e1-9ff3-4f1a-b4ff-b43c7da8f7f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:51.428141 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd04e1-9ff3-4f1a-b4ff-b43c7da8f7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:51.428430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd04e1-9ff3-4f1a-b4ff-b43c7da8f7f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:51.429054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd04e1-9ff3-4f1a-b4ff-b43c7da8f7f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:02:51.429355 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd04e1-9ff3-4f1a-b4ff-b43c7da8f7f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:51.430967 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd04e1-9ff3-4f1a-b4ff-b43c7da8f7f5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:51.432897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4984b-0672-4549-bcaa-3124d40a8d9d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:51.438244 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9f06a-ae20-4b67-838d-78c8115e6af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:51.438862 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9f06a-ae20-4b67-838d-78c8115e6af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:51.439133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9f06a-ae20-4b67-838d-78c8115e6af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:51.439453 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9f06a-ae20-4b67-838d-78c8115e6af0 admin admin] PATH_INFO: `/v3/users/87ea3a4618a34e588d5fd2159f6bc6c0` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:51.440623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9f06a-ae20-4b67-838d-78c8115e6af0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87ea3a4618a34e588d5fd2159f6bc6c0)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:51.442492 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9f06a-ae20-4b67-838d-78c8115e6af0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:51.442898 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1323/2650] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:02:51.447015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd04e1-9ff3-4f1a-b4ff-b43c7da8f7f5 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:51.457023 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd04e1-9ff3-4f1a-b4ff-b43c7da8f7f5 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:51.461631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0811f3-5a76-4b29-8d61-e323ac5d4ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:51.462208 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0811f3-5a76-4b29-8d61-e323ac5d4ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:51.462488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0811f3-5a76-4b29-8d61-e323ac5d4ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:51.462760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0811f3-5a76-4b29-8d61-e323ac5d4ba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:51.463378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0811f3-5a76-4b29-8d61-e323ac5d4ba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:51.465359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0811f3-5a76-4b29-8d61-e323ac5d4ba8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:51.490924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4fd9f06a-ae20-4b67-838d-78c8115e6af0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87ea3a4618a34e588d5fd2159f6bc6c0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:51.494484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1324/2651] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:51 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:51.498841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0811f3-5a76-4b29-8d61-e323ac5d4ba8 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:51.504516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=81,keystone:INSERT=2,keystone:DELETE=17 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:51.505917 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4fd9f06a-ae20-4b67-838d-78c8115e6af0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87ea3a4618a34e588d5fd2159f6bc6c0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:51.514947 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4fd9f06a-ae20-4b67-838d-78c8115e6af0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87ea3a4618a34e588d5fd2159f6bc6c0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:51.515584 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4fd9f06a-ae20-4b67-838d-78c8115e6af0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87ea3a4618a34e588d5fd2159f6bc6c0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:51.523202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4fd9f06a-ae20-4b67-838d-78c8115e6af0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87ea3a4618a34e588d5fd2159f6bc6c0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:51.530244 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1328/2652] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:51 2022] DELETE /identity/v3/users/87ea3a4618a34e588d5fd2159f6bc6c0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:51.534298 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ace65-bc2c-4877-bdb1-648210c94c2d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:51.534586 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ace65-bc2c-4877-bdb1-648210c94c2d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:51.534989 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ace65-bc2c-4877-bdb1-648210c94c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:51.553950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2f6ace65-bc2c-4877-bdb1-648210c94c2d None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:51.555164 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2f6ace65-bc2c-4877-bdb1-648210c94c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:51.562714 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0811f3-5a76-4b29-8d61-e323ac5d4ba8 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:51.614701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1329/2653] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:51.619971 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd49af6-d429-436f-bbda-330094b6d9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:51.620685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd49af6-d429-436f-bbda-330094b6d9c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:51.621024 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd49af6-d429-436f-bbda-330094b6d9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:51.621357 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd49af6-d429-436f-bbda-330094b6d9c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:51.622039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd49af6-d429-436f-bbda-330094b6d9c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:51.624471 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd49af6-d429-436f-bbda-330094b6d9c6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:51.624625 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ace65-bc2c-4877-bdb1-648210c94c2d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:51.626109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2f6ace65-bc2c-4877-bdb1-648210c94c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:51.650798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1325/2654] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:51 2022] POST /identity/v3/users => generated 297 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:51.651832 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd49af6-d429-436f-bbda-330094b6d9c6 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:51.653469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ebd49af6-d429-436f-bbda-330094b6d9c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:51.665932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd49af6-d429-436f-bbda-330094b6d9c6 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:51.671070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de62a9-c228-470c-922f-8a99e16bd82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:51.671672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de62a9-c228-470c-922f-8a99e16bd82e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:51.672473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de62a9-c228-470c-922f-8a99e16bd82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:51.672473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de62a9-c228-470c-922f-8a99e16bd82e admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:51.672856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de62a9-c228-470c-922f-8a99e16bd82e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:51.674470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de62a9-c228-470c-922f-8a99e16bd82e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:51.685819 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1326/2655] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:51.690147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de62a9-c228-470c-922f-8a99e16bd82e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:51.694608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ace65-bc2c-4877-bdb1-648210c94c2d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:51.699219 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317959dd-332f-4e8b-bbfd-174c929bebfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:51.700003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317959dd-332f-4e8b-bbfd-174c929bebfd service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:51.700595 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317959dd-332f-4e8b-bbfd-174c929bebfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:51.701749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de62a9-c228-470c-922f-8a99e16bd82e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:51.701985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317959dd-332f-4e8b-bbfd-174c929bebfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:51.706504 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faef725-332c-4a56-b3fb-a8c87aed6078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:51.707099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faef725-332c-4a56-b3fb-a8c87aed6078 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:51.707378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faef725-332c-4a56-b3fb-a8c87aed6078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:51.707652 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faef725-332c-4a56-b3fb-a8c87aed6078 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:51.708306 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faef725-332c-4a56-b3fb-a8c87aed6078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:51.709918 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faef725-332c-4a56-b3fb-a8c87aed6078 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:51.712780 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317959dd-332f-4e8b-bbfd-174c929bebfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:51.714357 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317959dd-332f-4e8b-bbfd-174c929bebfd service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:51.723146 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1327/2656] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:51.725365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1330/2657] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:02:51.727140 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faef725-332c-4a56-b3fb-a8c87aed6078 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:51.737749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faef725-332c-4a56-b3fb-a8c87aed6078 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:51.742815 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad332a-78f0-4b78-8a54-9ec52c4e83de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:51.743621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad332a-78f0-4b78-8a54-9ec52c4e83de admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:51.744025 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad332a-78f0-4b78-8a54-9ec52c4e83de admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:51.744488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad332a-78f0-4b78-8a54-9ec52c4e83de admin admin] PATH_INFO: `/v3/projects/f96efd445e864e35895c4164568bfc9b/users/052797fa05094c56a02b205f976c0278/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:51.761713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad332a-78f0-4b78-8a54-9ec52c4e83de admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96efd445e864e35895c4164568bfc9b, user_id=052797fa05094c56a02b205f976c0278, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:51.765532 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad332a-78f0-4b78-8a54-9ec52c4e83de admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:51.781638 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1328/2658] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:51 2022] PUT /identity/v3/projects/f96efd445e864e35895c4164568bfc9b/users/052797fa05094c56a02b205f976c0278/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:51.792399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385f8ec-c658-4d55-9f1f-44d307ec39d4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:51.792939 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385f8ec-c658-4d55-9f1f-44d307ec39d4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:51.793543 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385f8ec-c658-4d55-9f1f-44d307ec39d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:51.825730 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0385f8ec-c658-4d55-9f1f-44d307ec39d4 None None] MFA Rules not processed for user `052797fa05094c56a02b205f976c0278`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:51.827914 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0385f8ec-c658-4d55-9f1f-44d307ec39d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:51.918114 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad332a-78f0-4b78-8a54-9ec52c4e83de admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:51.920668 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1331/2659] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:51.936236 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:52.022148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad332a-78f0-4b78-8a54-9ec52c4e83de admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:52.038937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447c582-cf47-41b4-aa66-b9ef8cbb53fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:52.040689 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447c582-cf47-41b4-aa66-b9ef8cbb53fb admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:52.041522 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447c582-cf47-41b4-aa66-b9ef8cbb53fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:52.042152 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447c582-cf47-41b4-aa66-b9ef8cbb53fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:52.043413 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447c582-cf47-41b4-aa66-b9ef8cbb53fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:52.046030 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447c582-cf47-41b4-aa66-b9ef8cbb53fb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:52.057457 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385f8ec-c658-4d55-9f1f-44d307ec39d4 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:52.059148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0385f8ec-c658-4d55-9f1f-44d307ec39d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:52.089896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385f8ec-c658-4d55-9f1f-44d307ec39d4 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:52.100804 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1c704-0147-4e15-8fcc-012260c0c6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:52.101385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1c704-0147-4e15-8fcc-012260c0c6e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:52.101663 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1c704-0147-4e15-8fcc-012260c0c6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:52.103097 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1c704-0147-4e15-8fcc-012260c0c6e4 admin admin] PATH_INFO: `/v3/projects/72983346d12c425e80e9e187f84a0990` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:52.105922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1c704-0147-4e15-8fcc-012260c0c6e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72983346d12c425e80e9e187f84a0990)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:52.114897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1329/2660] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:51 2022] POST /identity/v3/projects => generated 393 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:02:52.116298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447c582-cf47-41b4-aa66-b9ef8cbb53fb admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:52.117035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1c704-0147-4e15-8fcc-012260c0c6e4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:52.137855 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:52.189106 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e8d1c704-0147-4e15-8fcc-012260c0c6e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72983346d12c425e80e9e187f84a0990', 'request_id': 'req-e8d1c704-0147-4e15-8fcc-012260c0c6e4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:52.196110 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e8d1c704-0147-4e15-8fcc-012260c0c6e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72983346d12c425e80e9e187f84a0990', 'request_id': 'req-e8d1c704-0147-4e15-8fcc-012260c0c6e4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:52.205405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e8d1c704-0147-4e15-8fcc-012260c0c6e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72983346d12c425e80e9e187f84a0990', 'request_id': 'req-e8d1c704-0147-4e15-8fcc-012260c0c6e4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:52.230023 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447c582-cf47-41b4-aa66-b9ef8cbb53fb admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:52.230253 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e8d1c704-0147-4e15-8fcc-012260c0c6e4 admin admin] The token cache is being invalidate because project 72983346d12c425e80e9e187f84a0990 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:52.230420 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e8d1c704-0147-4e15-8fcc-012260c0c6e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:52.231064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e8d1c704-0147-4e15-8fcc-012260c0c6e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:52.233481 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1332/2661] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:52 2022] DELETE /identity/v3/projects/72983346d12c425e80e9e187f84a0990 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:52.238396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1c704-0147-4e15-8fcc-012260c0c6e4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:52.241195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e8d1c704-0147-4e15-8fcc-012260c0c6e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:52.336475 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1c704-0147-4e15-8fcc-012260c0c6e4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:52.344103 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f3006-15bd-48c0-9c00-a79978656e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:52.345077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f3006-15bd-48c0-9c00-a79978656e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:52.345574 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f3006-15bd-48c0-9c00-a79978656e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:52.346084 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f3006-15bd-48c0-9c00-a79978656e49 admin admin] PATH_INFO: `/v3/users/75eb6da42b1e4223b29e01c827489bf5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:52.347479 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f3006-15bd-48c0-9c00-a79978656e49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75eb6da42b1e4223b29e01c827489bf5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:52.350524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f3006-15bd-48c0-9c00-a79978656e49 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:52.356878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ab19b-5027-44d1-9a72-6ef69a4977ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:52.358013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ab19b-5027-44d1-9a72-6ef69a4977ea admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:52.358235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ab19b-5027-44d1-9a72-6ef69a4977ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:52.358508 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ab19b-5027-44d1-9a72-6ef69a4977ea admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:52.359145 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ab19b-5027-44d1-9a72-6ef69a4977ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:52.361477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ab19b-5027-44d1-9a72-6ef69a4977ea admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:52.439434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1330/2662] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:52 2022] POST /identity/v3/users => generated 303 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:52.443731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ab19b-5027-44d1-9a72-6ef69a4977ea admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:52.446650 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-af5ab19b-5027-44d1-9a72-6ef69a4977ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:52.462521 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ab19b-5027-44d1-9a72-6ef69a4977ea admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:52.471798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9ec3e-eff7-4fcc-9fec-33f56ef916f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:52.472519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9ec3e-eff7-4fcc-9fec-33f56ef916f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:52.472798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9ec3e-eff7-4fcc-9fec-33f56ef916f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:52.473165 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9ec3e-eff7-4fcc-9fec-33f56ef916f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:52.473861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9ec3e-eff7-4fcc-9fec-33f56ef916f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:52.475595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9ec3e-eff7-4fcc-9fec-33f56ef916f6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:52.479250 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-623f3006-15bd-48c0-9c00-a79978656e49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75eb6da42b1e4223b29e01c827489bf5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:52.489004 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1331/2663] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:52.492805 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9ec3e-eff7-4fcc-9fec-33f56ef916f6 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:52.497370 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-623f3006-15bd-48c0-9c00-a79978656e49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75eb6da42b1e4223b29e01c827489bf5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:52.504430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9ec3e-eff7-4fcc-9fec-33f56ef916f6 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:52.507828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-623f3006-15bd-48c0-9c00-a79978656e49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75eb6da42b1e4223b29e01c827489bf5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:52.508501 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-623f3006-15bd-48c0-9c00-a79978656e49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75eb6da42b1e4223b29e01c827489bf5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:52.509346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1603514-9bd1-48f3-aa6c-a924e4e0384d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:52.510002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1603514-9bd1-48f3-aa6c-a924e4e0384d admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:52.510304 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1603514-9bd1-48f3-aa6c-a924e4e0384d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:52.510853 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1603514-9bd1-48f3-aa6c-a924e4e0384d admin admin] PATH_INFO: `/v3/projects/d1abd97842d644218d18efc62bea1ca3/users/36446cfa677e43498a2b2eeb1c61ff2e/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:52.518638 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-623f3006-15bd-48c0-9c00-a79978656e49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75eb6da42b1e4223b29e01c827489bf5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:52.524982 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1333/2664] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:52 2022] DELETE /identity/v3/users/75eb6da42b1e4223b29e01c827489bf5 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:52.529934 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1603514-9bd1-48f3-aa6c-a924e4e0384d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1abd97842d644218d18efc62bea1ca3, user_id=36446cfa677e43498a2b2eeb1c61ff2e, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:52.532739 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1603514-9bd1-48f3-aa6c-a924e4e0384d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:52.540960 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:52.550034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1332/2665] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:52 2022] PUT /identity/v3/projects/d1abd97842d644218d18efc62bea1ca3/users/36446cfa677e43498a2b2eeb1c61ff2e/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:52.553744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f3006-15bd-48c0-9c00-a79978656e49 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:52.555169 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-623f3006-15bd-48c0-9c00-a79978656e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:52.574593 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:52.624147 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f3006-15bd-48c0-9c00-a79978656e49 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:52.690337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00cf41a-76fd-450e-8ddf-68205d4753df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:52.691013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cf41a-76fd-450e-8ddf-68205d4753df admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:52.691373 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cf41a-76fd-450e-8ddf-68205d4753df admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:52.691547 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cf41a-76fd-450e-8ddf-68205d4753df admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:52.692249 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00cf41a-76fd-450e-8ddf-68205d4753df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:52.694131 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00cf41a-76fd-450e-8ddf-68205d4753df admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:52.704815 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1334/2666] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:52.709018 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1603514-9bd1-48f3-aa6c-a924e4e0384d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:52.720218 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1603514-9bd1-48f3-aa6c-a924e4e0384d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:52.725078 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251e467-fcf2-4312-9f5b-019497ff8716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:52.725737 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251e467-fcf2-4312-9f5b-019497ff8716 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:52.726014 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251e467-fcf2-4312-9f5b-019497ff8716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:52.726286 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251e467-fcf2-4312-9f5b-019497ff8716 admin admin] PATH_INFO: `/v3/projects/d1abd97842d644218d18efc62bea1ca3/users/36446cfa677e43498a2b2eeb1c61ff2e/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:52.728274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251e467-fcf2-4312-9f5b-019497ff8716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1abd97842d644218d18efc62bea1ca3, user_id=36446cfa677e43498a2b2eeb1c61ff2e, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:52.731078 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251e467-fcf2-4312-9f5b-019497ff8716 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:52.743053 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1333/2667] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:52 2022] PUT /identity/v3/projects/d1abd97842d644218d18efc62bea1ca3/users/36446cfa677e43498a2b2eeb1c61ff2e/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:52.754808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0285c2b-d2a0-4e40-ab9a-1e02a2c5dbdf None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:52.755092 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0285c2b-d2a0-4e40-ab9a-1e02a2c5dbdf None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:52.755363 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0285c2b-d2a0-4e40-ab9a-1e02a2c5dbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:52.786360 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a0285c2b-d2a0-4e40-ab9a-1e02a2c5dbdf None None] MFA Rules not processed for user `36446cfa677e43498a2b2eeb1c61ff2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:52.787735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a0285c2b-d2a0-4e40-ab9a-1e02a2c5dbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:52.819262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251e467-fcf2-4312-9f5b-019497ff8716 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:52.821785 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1251e467-fcf2-4312-9f5b-019497ff8716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:52.881844 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1335/2668] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:52 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:52.893292 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0285c2b-d2a0-4e40-ab9a-1e02a2c5dbdf None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:52.894772 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a0285c2b-d2a0-4e40-ab9a-1e02a2c5dbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:52.915402 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251e467-fcf2-4312-9f5b-019497ff8716 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:52.921313 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:52.922204 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:52.922640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:52.923056 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c admin admin] PATH_INFO: `/v3/projects/f3480e291d0a4346b320f2a22ef88d73` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:52.924412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3480e291d0a4346b320f2a22ef88d73)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:52.927065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:52.951913 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:52.969381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0285c2b-d2a0-4e40-ab9a-1e02a2c5dbdf None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:52.979486 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3480e291d0a4346b320f2a22ef88d73', 'request_id': 'req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:52.985467 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3480e291d0a4346b320f2a22ef88d73', 'request_id': 'req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:52.991945 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3480e291d0a4346b320f2a22ef88d73', 'request_id': 'req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:53.005566 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c admin admin] The token cache is being invalidate because project f3480e291d0a4346b320f2a22ef88d73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:53.005838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:53.006469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:53.007956 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1334/2669] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:52 2022] DELETE /identity/v3/projects/f3480e291d0a4346b320f2a22ef88d73 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:53.012010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:53.014391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:53.021105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06acb5b-da2a-4b0d-9f03-126fcc01ed1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:53.021860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06acb5b-da2a-4b0d-9f03-126fcc01ed1d service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:53.022143 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06acb5b-da2a-4b0d-9f03-126fcc01ed1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:53.022417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06acb5b-da2a-4b0d-9f03-126fcc01ed1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:53.024204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c06acb5b-da2a-4b0d-9f03-126fcc01ed1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:53.085848 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0895b2e-6ced-462f-ba3e-4ec8a3f2ce0c admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:53.090523 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dcd6fa-3965-4ac8-bd7c-e320230bd1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:53.091184 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dcd6fa-3965-4ac8-bd7c-e320230bd1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:53.092197 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dcd6fa-3965-4ac8-bd7c-e320230bd1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:53.092691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dcd6fa-3965-4ac8-bd7c-e320230bd1e3 admin admin] PATH_INFO: `/v3/users/98732edd40494bbda47ae05b86dd2515` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:53.093863 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dcd6fa-3965-4ac8-bd7c-e320230bd1e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98732edd40494bbda47ae05b86dd2515)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:53.095334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06acb5b-da2a-4b0d-9f03-126fcc01ed1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:53.095777 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dcd6fa-3965-4ac8-bd7c-e320230bd1e3 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:53.096797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06acb5b-da2a-4b0d-9f03-126fcc01ed1d service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:53.106691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1336/2670] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:52 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:02:53.112605 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:53.147005 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-39dcd6fa-3965-4ac8-bd7c-e320230bd1e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98732edd40494bbda47ae05b86dd2515'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:53.147671 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-39dcd6fa-3965-4ac8-bd7c-e320230bd1e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98732edd40494bbda47ae05b86dd2515'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:53.159778 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-39dcd6fa-3965-4ac8-bd7c-e320230bd1e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98732edd40494bbda47ae05b86dd2515'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:53.160509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-39dcd6fa-3965-4ac8-bd7c-e320230bd1e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98732edd40494bbda47ae05b86dd2515'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:53.165438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06acb5b-da2a-4b0d-9f03-126fcc01ed1d service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:53.167658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c06acb5b-da2a-4b0d-9f03-126fcc01ed1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:53.167914 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-39dcd6fa-3965-4ac8-bd7c-e320230bd1e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98732edd40494bbda47ae05b86dd2515'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:53.180304 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06acb5b-da2a-4b0d-9f03-126fcc01ed1d service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:53.182541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1335/2671] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:53 2022] DELETE /identity/v3/users/98732edd40494bbda47ae05b86dd2515 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:53.186490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3736b3b-0cb2-44b2-a820-5f896ead89c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:53.187360 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3736b3b-0cb2-44b2-a820-5f896ead89c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:53.187638 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3736b3b-0cb2-44b2-a820-5f896ead89c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:53.187910 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3736b3b-0cb2-44b2-a820-5f896ead89c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:53.189765 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b3736b3b-0cb2-44b2-a820-5f896ead89c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:53.251371 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3736b3b-0cb2-44b2-a820-5f896ead89c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:53.252828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3736b3b-0cb2-44b2-a820-5f896ead89c8 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:53.262327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1337/2672] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:53 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:53.462955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713c805-d672-4dea-86cb-6eac87880769 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:53.463331 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713c805-d672-4dea-86cb-6eac87880769 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:53.463657 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713c805-d672-4dea-86cb-6eac87880769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:53.486905 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b713c805-d672-4dea-86cb-6eac87880769 None None] MFA Rules not processed for user `7a67efdb6af84cfb92df1be6d8345ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:53.488217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b713c805-d672-4dea-86cb-6eac87880769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:53.510278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3736b3b-0cb2-44b2-a820-5f896ead89c8 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:53.511917 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b3736b3b-0cb2-44b2-a820-5f896ead89c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:53.534500 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3736b3b-0cb2-44b2-a820-5f896ead89c8 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:53.602425 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1336/2673] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:53.611695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:53.650968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d1f0c-518c-4434-9a23-66867916ca12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:53.651645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d1f0c-518c-4434-9a23-66867916ca12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:53.652172 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d1f0c-518c-4434-9a23-66867916ca12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:53.652506 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d1f0c-518c-4434-9a23-66867916ca12 admin admin] PATH_INFO: `/v3/projects/21e26ba936834316b7a6d5e3f1796509` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:53.654437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d1f0c-518c-4434-9a23-66867916ca12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21e26ba936834316b7a6d5e3f1796509)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:53.661528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d1f0c-518c-4434-9a23-66867916ca12 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:53.728621 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8c9d1f0c-518c-4434-9a23-66867916ca12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21e26ba936834316b7a6d5e3f1796509', 'request_id': 'req-8c9d1f0c-518c-4434-9a23-66867916ca12'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:53.734778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8c9d1f0c-518c-4434-9a23-66867916ca12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21e26ba936834316b7a6d5e3f1796509', 'request_id': 'req-8c9d1f0c-518c-4434-9a23-66867916ca12'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:53.741254 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8c9d1f0c-518c-4434-9a23-66867916ca12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21e26ba936834316b7a6d5e3f1796509', 'request_id': 'req-8c9d1f0c-518c-4434-9a23-66867916ca12'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:53.755625 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8c9d1f0c-518c-4434-9a23-66867916ca12 admin admin] The token cache is being invalidate because project 21e26ba936834316b7a6d5e3f1796509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:53.756599 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8c9d1f0c-518c-4434-9a23-66867916ca12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:53.757294 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8c9d1f0c-518c-4434-9a23-66867916ca12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:53.759096 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1338/2674] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:53 2022] DELETE /identity/v3/projects/21e26ba936834316b7a6d5e3f1796509 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:56.756334 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713c805-d672-4dea-86cb-6eac87880769 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:56.757746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b713c805-d672-4dea-86cb-6eac87880769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:56.772105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713c805-d672-4dea-86cb-6eac87880769 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:56.830044 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee8058-1286-4a91-8cab-5a554d8d9adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:56.830560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee8058-1286-4a91-8cab-5a554d8d9adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:56.830872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee8058-1286-4a91-8cab-5a554d8d9adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:56.831115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee8058-1286-4a91-8cab-5a554d8d9adc admin admin] PATH_INFO: `/v3/users/052797fa05094c56a02b205f976c0278` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:56.832294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee8058-1286-4a91-8cab-5a554d8d9adc admin admin] RBAC: Authorizing `identity:delete_user(user_id=052797fa05094c56a02b205f976c0278)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:56.834218 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee8058-1286-4a91-8cab-5a554d8d9adc admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:56.884801 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:56.903525 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fbee8058-1286-4a91-8cab-5a554d8d9adc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '052797fa05094c56a02b205f976c0278'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:56.905121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fbee8058-1286-4a91-8cab-5a554d8d9adc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '052797fa05094c56a02b205f976c0278'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:56.919217 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fbee8058-1286-4a91-8cab-5a554d8d9adc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '052797fa05094c56a02b205f976c0278'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:56.920009 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fbee8058-1286-4a91-8cab-5a554d8d9adc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '052797fa05094c56a02b205f976c0278'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:56.927464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fbee8058-1286-4a91-8cab-5a554d8d9adc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '052797fa05094c56a02b205f976c0278'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:56.941289 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1337/2675] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:56 2022] DELETE /identity/v3/users/052797fa05094c56a02b205f976c0278 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:56.945887 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7bcc6-db30-40e6-a8ea-4dd94d4b707e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:56.946419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7bcc6-db30-40e6-a8ea-4dd94d4b707e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:56.946419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7bcc6-db30-40e6-a8ea-4dd94d4b707e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:56.966169 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-eda7bcc6-db30-40e6-a8ea-4dd94d4b707e None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:56.967657 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-eda7bcc6-db30-40e6-a8ea-4dd94d4b707e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:57.029873 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1339/2676] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:57.039395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee8058-1286-4a91-8cab-5a554d8d9adc admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:57.040838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fbee8058-1286-4a91-8cab-5a554d8d9adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:57.100548 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee8058-1286-4a91-8cab-5a554d8d9adc admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:57.104229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb001ba0-0689-4153-8bd4-620675d06760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:57.104840 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb001ba0-0689-4153-8bd4-620675d06760 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:57.105140 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb001ba0-0689-4153-8bd4-620675d06760 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:57.105399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb001ba0-0689-4153-8bd4-620675d06760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:57.115144 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb001ba0-0689-4153-8bd4-620675d06760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:57.116621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb001ba0-0689-4153-8bd4-620675d06760 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:57.128178 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1338/2677] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:57.418933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7bcc6-db30-40e6-a8ea-4dd94d4b707e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:57.420949 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-eda7bcc6-db30-40e6-a8ea-4dd94d4b707e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:57.433345 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7bcc6-db30-40e6-a8ea-4dd94d4b707e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:57.439217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:57.439861 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:57.440171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:57.440446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f admin admin] PATH_INFO: `/v3/projects/f96efd445e864e35895c4164568bfc9b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:57.441608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96efd445e864e35895c4164568bfc9b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:57.443535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:57.496789 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96efd445e864e35895c4164568bfc9b', 'request_id': 'req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:57.502838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96efd445e864e35895c4164568bfc9b', 'request_id': 'req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:57.508858 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96efd445e864e35895c4164568bfc9b', 'request_id': 'req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:57.520890 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f admin admin] The token cache is being invalidate because project f96efd445e864e35895c4164568bfc9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:57.521171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:57.521796 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:57.523381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1340/2678] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:57 2022] DELETE /identity/v3/projects/f96efd445e864e35895c4164568bfc9b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:57.527488 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb001ba0-0689-4153-8bd4-620675d06760 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:57.537652 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb001ba0-0689-4153-8bd4-620675d06760 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:57.597402 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba9769-0897-49db-a245-b7c2be5353b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:57.598103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba9769-0897-49db-a245-b7c2be5353b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:57.598445 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba9769-0897-49db-a245-b7c2be5353b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:57.598717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba9769-0897-49db-a245-b7c2be5353b0 admin admin] PATH_INFO: `/v3/users/36446cfa677e43498a2b2eeb1c61ff2e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:57.599825 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba9769-0897-49db-a245-b7c2be5353b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36446cfa677e43498a2b2eeb1c61ff2e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:57.601749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba9769-0897-49db-a245-b7c2be5353b0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:57.654802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d4ba9769-0897-49db-a245-b7c2be5353b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36446cfa677e43498a2b2eeb1c61ff2e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:57.655864 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d4ba9769-0897-49db-a245-b7c2be5353b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36446cfa677e43498a2b2eeb1c61ff2e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:57.669979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:57.671572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d4ba9769-0897-49db-a245-b7c2be5353b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36446cfa677e43498a2b2eeb1c61ff2e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:57.672977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d4ba9769-0897-49db-a245-b7c2be5353b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36446cfa677e43498a2b2eeb1c61ff2e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:57.682188 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d4ba9769-0897-49db-a245-b7c2be5353b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36446cfa677e43498a2b2eeb1c61ff2e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:57.699048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1339/2679] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:57 2022] DELETE /identity/v3/users/36446cfa677e43498a2b2eeb1c61ff2e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:57.989614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:57.991060 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:57.999788 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:58.005397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b99df-4ab9-45d8-912d-51a7e8cbdf2f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:58.071941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19527dd4-8a68-4267-a0e0-ed4f4775a86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:58.072606 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19527dd4-8a68-4267-a0e0-ed4f4775a86a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:58.072883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19527dd4-8a68-4267-a0e0-ed4f4775a86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:58.073155 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19527dd4-8a68-4267-a0e0-ed4f4775a86a admin admin] PATH_INFO: `/v3/projects/d1abd97842d644218d18efc62bea1ca3` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:58.074263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19527dd4-8a68-4267-a0e0-ed4f4775a86a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1abd97842d644218d18efc62bea1ca3)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:58.077036 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19527dd4-8a68-4267-a0e0-ed4f4775a86a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:58.123284 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-19527dd4-8a68-4267-a0e0-ed4f4775a86a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1abd97842d644218d18efc62bea1ca3', 'request_id': 'req-19527dd4-8a68-4267-a0e0-ed4f4775a86a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:58.129170 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-19527dd4-8a68-4267-a0e0-ed4f4775a86a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1abd97842d644218d18efc62bea1ca3', 'request_id': 'req-19527dd4-8a68-4267-a0e0-ed4f4775a86a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:58.135366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-19527dd4-8a68-4267-a0e0-ed4f4775a86a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1abd97842d644218d18efc62bea1ca3', 'request_id': 'req-19527dd4-8a68-4267-a0e0-ed4f4775a86a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:58.147529 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-19527dd4-8a68-4267-a0e0-ed4f4775a86a admin admin] The token cache is being invalidate because project d1abd97842d644218d18efc62bea1ca3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:02:58.147818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-19527dd4-8a68-4267-a0e0-ed4f4775a86a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:58.148455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-19527dd4-8a68-4267-a0e0-ed4f4775a86a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:58.149925 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1341/2680] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:02:57 2022] DELETE /identity/v3/projects/d1abd97842d644218d18efc62bea1ca3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:58.154970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326be71-d75b-46a9-bd15-22fba29130d0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:58.155479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326be71-d75b-46a9-bd15-22fba29130d0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:58.155589 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326be71-d75b-46a9-bd15-22fba29130d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:58.175422 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-5326be71-d75b-46a9-bd15-22fba29130d0 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:58.176707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5326be71-d75b-46a9-bd15-22fba29130d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:58.238240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1340/2681] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:58.245774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39888ac2-959e-4f9e-a903-ef9ffe74572e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:58.246057 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39888ac2-959e-4f9e-a903-ef9ffe74572e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:58.246327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39888ac2-959e-4f9e-a903-ef9ffe74572e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:58.266533 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-39888ac2-959e-4f9e-a903-ef9ffe74572e None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:58.267922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-39888ac2-959e-4f9e-a903-ef9ffe74572e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:58.276116 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1342/2682] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:58.281274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326be71-d75b-46a9-bd15-22fba29130d0 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:58.291777 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326be71-d75b-46a9-bd15-22fba29130d0 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:58.296787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3525c7-4ef2-4ed1-a7af-32644449c086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:58.297404 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3525c7-4ef2-4ed1-a7af-32644449c086 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:58.297791 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3525c7-4ef2-4ed1-a7af-32644449c086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:58.298066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3525c7-4ef2-4ed1-a7af-32644449c086 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:58.298748 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3525c7-4ef2-4ed1-a7af-32644449c086 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:02:58.299049 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3525c7-4ef2-4ed1-a7af-32644449c086 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:58.300760 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3525c7-4ef2-4ed1-a7af-32644449c086 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:58.313320 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1341/2683] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:02:58.317624 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39888ac2-959e-4f9e-a903-ef9ffe74572e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:58.329220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39888ac2-959e-4f9e-a903-ef9ffe74572e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:58.334924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2b1e4-5705-462b-9191-511e286e0317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:58.335516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2b1e4-5705-462b-9191-511e286e0317 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:58.335788 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2b1e4-5705-462b-9191-511e286e0317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:58.336053 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2b1e4-5705-462b-9191-511e286e0317 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:58.336718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2b1e4-5705-462b-9191-511e286e0317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:58.338621 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2b1e4-5705-462b-9191-511e286e0317 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:58.373583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1343/2684] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:58 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:58.374811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3525c7-4ef2-4ed1-a7af-32644449c086 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:58.391528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3525c7-4ef2-4ed1-a7af-32644449c086 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:58.460829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725edb3-331d-4290-ab00-02b52ea556b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:58.461642 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725edb3-331d-4290-ab00-02b52ea556b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:58.461925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725edb3-331d-4290-ab00-02b52ea556b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:58.462228 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725edb3-331d-4290-ab00-02b52ea556b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:58.463069 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725edb3-331d-4290-ab00-02b52ea556b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:58.465711 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725edb3-331d-4290-ab00-02b52ea556b6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:58.491580 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1342/2685] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:58 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:58.492842 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2b1e4-5705-462b-9191-511e286e0317 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:58.503419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2b1e4-5705-462b-9191-511e286e0317 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:58.508526 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a1ce3-3640-4416-8565-4cdd45c3a386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:58.509127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a1ce3-3640-4416-8565-4cdd45c3a386 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:58.509406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a1ce3-3640-4416-8565-4cdd45c3a386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:58.509707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a1ce3-3640-4416-8565-4cdd45c3a386 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:58.510396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a1ce3-3640-4416-8565-4cdd45c3a386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:58.511996 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a1ce3-3640-4416-8565-4cdd45c3a386 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:58.524586 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1344/2686] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:58.529216 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725edb3-331d-4290-ab00-02b52ea556b6 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:58.539482 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725edb3-331d-4290-ab00-02b52ea556b6 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:58.544145 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b700bc8-348f-44f8-8158-80b803af6f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:58.545600 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b700bc8-348f-44f8-8158-80b803af6f63 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:58.545865 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b700bc8-348f-44f8-8158-80b803af6f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:58.546130 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b700bc8-348f-44f8-8158-80b803af6f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:58.546768 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b700bc8-348f-44f8-8158-80b803af6f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:58.548412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b700bc8-348f-44f8-8158-80b803af6f63 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:58.561132 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1343/2687] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:58.565889 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a1ce3-3640-4416-8565-4cdd45c3a386 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:58.577906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a1ce3-3640-4416-8565-4cdd45c3a386 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:58.583451 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e60f3-cf61-449b-a821-b4f103c46c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:58.584288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e60f3-cf61-449b-a821-b4f103c46c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:58.584568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e60f3-cf61-449b-a821-b4f103c46c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:58.584835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e60f3-cf61-449b-a821-b4f103c46c65 admin admin] PATH_INFO: `/v3/projects/8d677351e7cb4f70a4348858d5a51e5a/users/3bf34b9f97cd499593dc157f00e075cd/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:58.601867 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e60f3-cf61-449b-a821-b4f103c46c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d677351e7cb4f70a4348858d5a51e5a, user_id=3bf34b9f97cd499593dc157f00e075cd, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:58.604818 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e60f3-cf61-449b-a821-b4f103c46c65 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:58.620028 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1345/2688] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:58 2022] PUT /identity/v3/projects/8d677351e7cb4f70a4348858d5a51e5a/users/3bf34b9f97cd499593dc157f00e075cd/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:58.630985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356aadb-3e48-4ee0-8b4c-5b244e4fe361 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:58.631268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356aadb-3e48-4ee0-8b4c-5b244e4fe361 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:58.631538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356aadb-3e48-4ee0-8b4c-5b244e4fe361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:58.659660 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3356aadb-3e48-4ee0-8b4c-5b244e4fe361 None None] MFA Rules not processed for user `3bf34b9f97cd499593dc157f00e075cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:58.660947 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3356aadb-3e48-4ee0-8b4c-5b244e4fe361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:58.699048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:58.735543 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1344/2689] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:58.739968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e60f3-cf61-449b-a821-b4f103c46c65 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:58.772757 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:58.805693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e60f3-cf61-449b-a821-b4f103c46c65 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:58.810429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace27f5a-0a96-42b6-ab98-2dc6f11b5d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:58.811024 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace27f5a-0a96-42b6-ab98-2dc6f11b5d33 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:58.811303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace27f5a-0a96-42b6-ab98-2dc6f11b5d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:58.811825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace27f5a-0a96-42b6-ab98-2dc6f11b5d33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:58.812493 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace27f5a-0a96-42b6-ab98-2dc6f11b5d33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:58.814370 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace27f5a-0a96-42b6-ab98-2dc6f11b5d33 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:58.840386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1346/2690] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:58 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:58.847144 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356aadb-3e48-4ee0-8b4c-5b244e4fe361 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:58.859444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356aadb-3e48-4ee0-8b4c-5b244e4fe361 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:58.920863 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cc5a9-50b3-4cd5-86f7-bc9b1933681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:58.921568 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cc5a9-50b3-4cd5-86f7-bc9b1933681d admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:58.921906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cc5a9-50b3-4cd5-86f7-bc9b1933681d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:58.922247 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cc5a9-50b3-4cd5-86f7-bc9b1933681d admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:58.922870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cc5a9-50b3-4cd5-86f7-bc9b1933681d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:58.924823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cc5a9-50b3-4cd5-86f7-bc9b1933681d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:58.946552 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1345/2691] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:58 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:58.951157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace27f5a-0a96-42b6-ab98-2dc6f11b5d33 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:58.961740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace27f5a-0a96-42b6-ab98-2dc6f11b5d33 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:58.966961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154269fd-0a66-42d1-98b7-1a2b9cbe89d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:58.967740 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154269fd-0a66-42d1-98b7-1a2b9cbe89d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:58.968019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154269fd-0a66-42d1-98b7-1a2b9cbe89d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:58.968334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154269fd-0a66-42d1-98b7-1a2b9cbe89d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:58.968972 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154269fd-0a66-42d1-98b7-1a2b9cbe89d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:58.970556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154269fd-0a66-42d1-98b7-1a2b9cbe89d9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:58.981168 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1347/2692] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:58.985305 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cc5a9-50b3-4cd5-86f7-bc9b1933681d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:58.995399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cc5a9-50b3-4cd5-86f7-bc9b1933681d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:59.000903 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f46425-159d-4b04-a167-41dc7a2e5b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:59.001506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f46425-159d-4b04-a167-41dc7a2e5b45 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:59.001779 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f46425-159d-4b04-a167-41dc7a2e5b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:59.002043 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f46425-159d-4b04-a167-41dc7a2e5b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:59.002703 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f46425-159d-4b04-a167-41dc7a2e5b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:59.004315 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f46425-159d-4b04-a167-41dc7a2e5b45 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:59.015402 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1346/2693] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:59.019641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154269fd-0a66-42d1-98b7-1a2b9cbe89d9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:59.030170 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154269fd-0a66-42d1-98b7-1a2b9cbe89d9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:59.035057 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37ddac-8018-4636-8462-d953d407c10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:59.035701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37ddac-8018-4636-8462-d953d407c10e admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:59.035977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37ddac-8018-4636-8462-d953d407c10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:59.036286 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37ddac-8018-4636-8462-d953d407c10e admin admin] PATH_INFO: `/v3/projects/0f1967ce6f4f4824ab8a46befea004cd/users/8f90574e18a04a17b515aa87e2e1ce48/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:59.051144 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37ddac-8018-4636-8462-d953d407c10e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f1967ce6f4f4824ab8a46befea004cd, user_id=8f90574e18a04a17b515aa87e2e1ce48, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:59.053794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37ddac-8018-4636-8462-d953d407c10e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:59.070619 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1348/2694] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:59 2022] PUT /identity/v3/projects/0f1967ce6f4f4824ab8a46befea004cd/users/8f90574e18a04a17b515aa87e2e1ce48/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:59.082534 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606fa30-3dab-4bc9-b8dd-7f61eef103a8 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:59.082817 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606fa30-3dab-4bc9-b8dd-7f61eef103a8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:59.083091 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606fa30-3dab-4bc9-b8dd-7f61eef103a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:59.115537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1606fa30-3dab-4bc9-b8dd-7f61eef103a8 None None] MFA Rules not processed for user `8f90574e18a04a17b515aa87e2e1ce48`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:59.116853 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1606fa30-3dab-4bc9-b8dd-7f61eef103a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:59.193969 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1347/2695] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:59.194891 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37ddac-8018-4636-8462-d953d407c10e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:59.264600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37ddac-8018-4636-8462-d953d407c10e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:59.270162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242f60b-27cd-428c-8fc6-78da87657bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:59.270774 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242f60b-27cd-428c-8fc6-78da87657bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:59.271056 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242f60b-27cd-428c-8fc6-78da87657bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:59.271332 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242f60b-27cd-428c-8fc6-78da87657bc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:59.271978 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242f60b-27cd-428c-8fc6-78da87657bc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:59.273906 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242f60b-27cd-428c-8fc6-78da87657bc1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:59.304148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1349/2696] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:02:59 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:59.306890 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606fa30-3dab-4bc9-b8dd-7f61eef103a8 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:59.319064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606fa30-3dab-4bc9-b8dd-7f61eef103a8 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:59.364317 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242f60b-27cd-428c-8fc6-78da87657bc1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:59.366018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2242f60b-27cd-428c-8fc6-78da87657bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:59.366830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:59.389492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffaf38-3c4e-4d35-94f7-2e252224c54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:59.390103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffaf38-3c4e-4d35-94f7-2e252224c54e admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:59.390382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffaf38-3c4e-4d35-94f7-2e252224c54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:59.390654 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffaf38-3c4e-4d35-94f7-2e252224c54e admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:59.391343 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffaf38-3c4e-4d35-94f7-2e252224c54e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:59.393317 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffaf38-3c4e-4d35-94f7-2e252224c54e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:59.421109 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1348/2697] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:02:59 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:02:59.425227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffaf38-3c4e-4d35-94f7-2e252224c54e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:59.432727 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242f60b-27cd-428c-8fc6-78da87657bc1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:59.435392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffaf38-3c4e-4d35-94f7-2e252224c54e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:59.438212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c1712-6e57-48a5-a0ac-8336b4bbe6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:59.438841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c1712-6e57-48a5-a0ac-8336b4bbe6e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:59.439140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c1712-6e57-48a5-a0ac-8336b4bbe6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:59.439417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c1712-6e57-48a5-a0ac-8336b4bbe6e4 admin admin] PATH_INFO: `/v3/users/e2c6d14a1af44afdaf8dafa7e69130d2` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:59.440563 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a275b0-1e7e-4853-b373-e8f472c66c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:59.440688 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c1712-6e57-48a5-a0ac-8336b4bbe6e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2c6d14a1af44afdaf8dafa7e69130d2)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:59.441144 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a275b0-1e7e-4853-b373-e8f472c66c58 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:59.441448 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a275b0-1e7e-4853-b373-e8f472c66c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:59.441718 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a275b0-1e7e-4853-b373-e8f472c66c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:59.442336 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a275b0-1e7e-4853-b373-e8f472c66c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:59.442535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c1712-6e57-48a5-a0ac-8336b4bbe6e4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:59.443961 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a275b0-1e7e-4853-b373-e8f472c66c58 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:59.447465 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:59.458425 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1349/2698] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:02:59.459482 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a275b0-1e7e-4853-b373-e8f472c66c58 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:59.469213 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a275b0-1e7e-4853-b373-e8f472c66c58 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:59.474415 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a2a6f1-0852-4e57-a40e-9df8db427b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:59.475154 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2a6f1-0852-4e57-a40e-9df8db427b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:59.475457 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2a6f1-0852-4e57-a40e-9df8db427b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:59.475824 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2a6f1-0852-4e57-a40e-9df8db427b9b admin admin] PATH_INFO: `/v3/projects/25864af38b7043718f67cde53c62ea1d/users/9c349a3f1a3e4b6e900177f8e90353e3/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:59.491079 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a2a6f1-0852-4e57-a40e-9df8db427b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=25864af38b7043718f67cde53c62ea1d, user_id=9c349a3f1a3e4b6e900177f8e90353e3, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:59.491418 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-814c1712-6e57-48a5-a0ac-8336b4bbe6e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2c6d14a1af44afdaf8dafa7e69130d2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:59.493999 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a2a6f1-0852-4e57-a40e-9df8db427b9b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:59.504838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-814c1712-6e57-48a5-a0ac-8336b4bbe6e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2c6d14a1af44afdaf8dafa7e69130d2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:59.507256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1350/2699] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:59 2022] PUT /identity/v3/projects/25864af38b7043718f67cde53c62ea1d/users/9c349a3f1a3e4b6e900177f8e90353e3/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:02:59.511205 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a2a6f1-0852-4e57-a40e-9df8db427b9b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:59.513780 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-814c1712-6e57-48a5-a0ac-8336b4bbe6e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2c6d14a1af44afdaf8dafa7e69130d2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:59.514466 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-814c1712-6e57-48a5-a0ac-8336b4bbe6e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2c6d14a1af44afdaf8dafa7e69130d2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:59.522113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-814c1712-6e57-48a5-a0ac-8336b4bbe6e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2c6d14a1af44afdaf8dafa7e69130d2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:02:59.529524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1350/2700] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:02:59 2022] DELETE /identity/v3/users/e2c6d14a1af44afdaf8dafa7e69130d2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:59.533848 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953e0f3-dc5a-461d-beef-fa25dc33f79c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:59.534158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953e0f3-dc5a-461d-beef-fa25dc33f79c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:59.534425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953e0f3-dc5a-461d-beef-fa25dc33f79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:59.553511 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8953e0f3-dc5a-461d-beef-fa25dc33f79c None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:59.555023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8953e0f3-dc5a-461d-beef-fa25dc33f79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:59.573724 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a2a6f1-0852-4e57-a40e-9df8db427b9b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:59.615320 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1351/2701] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:02:59.626012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953e0f3-dc5a-461d-beef-fa25dc33f79c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:59.627364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8953e0f3-dc5a-461d-beef-fa25dc33f79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:59.632261 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c52d81-49f8-48ce-aeba-fe0afddca761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:59.632877 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c52d81-49f8-48ce-aeba-fe0afddca761 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:59.633155 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c52d81-49f8-48ce-aeba-fe0afddca761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:59.633426 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c52d81-49f8-48ce-aeba-fe0afddca761 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:59.634052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c52d81-49f8-48ce-aeba-fe0afddca761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:59.635632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c52d81-49f8-48ce-aeba-fe0afddca761 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:59.645877 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1351/2702] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:02:59.649956 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c52d81-49f8-48ce-aeba-fe0afddca761 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:59.651649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e6c52d81-49f8-48ce-aeba-fe0afddca761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:59.663871 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c52d81-49f8-48ce-aeba-fe0afddca761 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:59.668569 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520860f5-efc4-4b55-991f-71cb8697edd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:59.669746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520860f5-efc4-4b55-991f-71cb8697edd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:59.670035 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520860f5-efc4-4b55-991f-71cb8697edd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:59.670484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520860f5-efc4-4b55-991f-71cb8697edd5 admin admin] PATH_INFO: `/v3/projects/25864af38b7043718f67cde53c62ea1d/users/9c349a3f1a3e4b6e900177f8e90353e3/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:59.672453 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520860f5-efc4-4b55-991f-71cb8697edd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25864af38b7043718f67cde53c62ea1d, user_id=9c349a3f1a3e4b6e900177f8e90353e3, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:59.675092 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520860f5-efc4-4b55-991f-71cb8697edd5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:59.682994 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953e0f3-dc5a-461d-beef-fa25dc33f79c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:59.686354 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1352/2703] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:02:59 2022] PUT /identity/v3/projects/25864af38b7043718f67cde53c62ea1d/users/9c349a3f1a3e4b6e900177f8e90353e3/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:02:59.696892 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecf195-395f-4a6e-a52c-e43600942b0f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:59.697176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecf195-395f-4a6e-a52c-e43600942b0f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:59.697447 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecf195-395f-4a6e-a52c-e43600942b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:59.720950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:59.727189 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-acecf195-395f-4a6e-a52c-e43600942b0f None None] MFA Rules not processed for user `9c349a3f1a3e4b6e900177f8e90353e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:02:59.728462 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-acecf195-395f-4a6e-a52c-e43600942b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:59.734390 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8c3c1-d938-4d36-a416-5854fbd92224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:59.735066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8c3c1-d938-4d36-a416-5854fbd92224 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:59.735414 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8c3c1-d938-4d36-a416-5854fbd92224 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:59.735682 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8c3c1-d938-4d36-a416-5854fbd92224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:59.746214 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:02:59.797264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8c3c1-d938-4d36-a416-5854fbd92224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:59.798593 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8c3c1-d938-4d36-a416-5854fbd92224 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:59.805884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1353/2704] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:02:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:02:59.808291 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1352/2705] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:02:59.818224 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecf195-395f-4a6e-a52c-e43600942b0f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:59.819607 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-acecf195-395f-4a6e-a52c-e43600942b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:59.879582 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecf195-395f-4a6e-a52c-e43600942b0f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:59.884215 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657cb73-06c0-474b-9816-944b4fb95176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:59.885247 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657cb73-06c0-474b-9816-944b4fb95176 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:59.885247 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657cb73-06c0-474b-9816-944b4fb95176 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:59.885574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657cb73-06c0-474b-9816-944b4fb95176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:59.895786 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657cb73-06c0-474b-9816-944b4fb95176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:02:59.897370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657cb73-06c0-474b-9816-944b4fb95176 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:02:59.908051 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1354/2706] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:02:59.950747 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8c3c1-d938-4d36-a416-5854fbd92224 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:02:59.952296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-50e8c3c1-d938-4d36-a416-5854fbd92224 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:02:59.965722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8c3c1-d938-4d36-a416-5854fbd92224 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:02:59.970145 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b27eae-18d7-40b9-96bf-ed4a68c9011a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:02:59.970831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b27eae-18d7-40b9-96bf-ed4a68c9011a service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:02:59.971181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b27eae-18d7-40b9-96bf-ed4a68c9011a service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:02:59.971517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b27eae-18d7-40b9-96bf-ed4a68c9011a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:02:59.973322 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e0b27eae-18d7-40b9-96bf-ed4a68c9011a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:00.036330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b27eae-18d7-40b9-96bf-ed4a68c9011a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:00.038146 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b27eae-18d7-40b9-96bf-ed4a68c9011a service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:00.049081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1353/2707] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:02:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:00.101087 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657cb73-06c0-474b-9816-944b4fb95176 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:00.102842 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c657cb73-06c0-474b-9816-944b4fb95176 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:00.135528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657cb73-06c0-474b-9816-944b4fb95176 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:00.148565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:00.149224 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:00.149475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:00.149742 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a admin admin] PATH_INFO: `/v3/projects/a0034c10c6404be19f19bb5c817c4807` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:00.152430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0034c10c6404be19f19bb5c817c4807)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:00.155193 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:00.244955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0034c10c6404be19f19bb5c817c4807', 'request_id': 'req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:00.251942 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0034c10c6404be19f19bb5c817c4807', 'request_id': 'req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:00.260001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0034c10c6404be19f19bb5c817c4807', 'request_id': 'req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:00.277347 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a admin admin] The token cache is being invalidate because project a0034c10c6404be19f19bb5c817c4807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:00.279048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:00.279966 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:00.281933 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1355/2708] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:00 2022] DELETE /identity/v3/projects/a0034c10c6404be19f19bb5c817c4807 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:00.286212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b27eae-18d7-40b9-96bf-ed4a68c9011a service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:00.298916 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b27eae-18d7-40b9-96bf-ed4a68c9011a service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:00.373528 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67771fa4-85d4-4b97-af87-cf1510aa58bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:00.374166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67771fa4-85d4-4b97-af87-cf1510aa58bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:00.374443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67771fa4-85d4-4b97-af87-cf1510aa58bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:00.374714 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67771fa4-85d4-4b97-af87-cf1510aa58bd admin admin] PATH_INFO: `/v3/users/a1365d05c215408790289cba9ee83f52` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:00.375852 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67771fa4-85d4-4b97-af87-cf1510aa58bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1365d05c215408790289cba9ee83f52)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:00.383285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67771fa4-85d4-4b97-af87-cf1510aa58bd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:00.454442 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:00.481003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-67771fa4-85d4-4b97-af87-cf1510aa58bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1365d05c215408790289cba9ee83f52'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:00.494766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-67771fa4-85d4-4b97-af87-cf1510aa58bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1365d05c215408790289cba9ee83f52'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:00.502323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-67771fa4-85d4-4b97-af87-cf1510aa58bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1365d05c215408790289cba9ee83f52'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:00.503014 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-67771fa4-85d4-4b97-af87-cf1510aa58bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1365d05c215408790289cba9ee83f52'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:00.510227 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-67771fa4-85d4-4b97-af87-cf1510aa58bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1365d05c215408790289cba9ee83f52'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:00.517331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1354/2709] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:03:00 2022] DELETE /identity/v3/users/a1365d05c215408790289cba9ee83f52 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:03:00.789168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:00.790694 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:00.804505 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c434f4b-ae8f-4e8e-a09e-7576d0db841a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:00.847396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:00.869952 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92e415-7305-4d88-be07-6706160aca66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:00.870578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92e415-7305-4d88-be07-6706160aca66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:00.870843 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92e415-7305-4d88-be07-6706160aca66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:00.871106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92e415-7305-4d88-be07-6706160aca66 admin admin] PATH_INFO: `/v3/projects/7ab347c217cb464c9e77fe12ead1dd78` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:00.872869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e92e415-7305-4d88-be07-6706160aca66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ab347c217cb464c9e77fe12ead1dd78)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:00.874788 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e92e415-7305-4d88-be07-6706160aca66 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:00.921450 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9e92e415-7305-4d88-be07-6706160aca66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ab347c217cb464c9e77fe12ead1dd78', 'request_id': 'req-9e92e415-7305-4d88-be07-6706160aca66'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:00.926882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9e92e415-7305-4d88-be07-6706160aca66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ab347c217cb464c9e77fe12ead1dd78', 'request_id': 'req-9e92e415-7305-4d88-be07-6706160aca66'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:00.933279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9e92e415-7305-4d88-be07-6706160aca66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ab347c217cb464c9e77fe12ead1dd78', 'request_id': 'req-9e92e415-7305-4d88-be07-6706160aca66'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:00.945910 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9e92e415-7305-4d88-be07-6706160aca66 admin admin] The token cache is being invalidate because project 7ab347c217cb464c9e77fe12ead1dd78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:00.946195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9e92e415-7305-4d88-be07-6706160aca66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:00.946803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9e92e415-7305-4d88-be07-6706160aca66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:00.948299 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1356/2710] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:00 2022] DELETE /identity/v3/projects/7ab347c217cb464c9e77fe12ead1dd78 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:00.954964 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd7133-a0a3-40df-bc3d-cff62ec062df None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:00.955248 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd7133-a0a3-40df-bc3d-cff62ec062df None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:00.955521 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd7133-a0a3-40df-bc3d-cff62ec062df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:00.975047 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e4fd7133-a0a3-40df-bc3d-cff62ec062df None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:00.976273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e4fd7133-a0a3-40df-bc3d-cff62ec062df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:01.034865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1355/2711] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:01.042640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca9ffa-3f40-49b2-9350-5577077b1207 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:01.042927 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca9ffa-3f40-49b2-9350-5577077b1207 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:01.043195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca9ffa-3f40-49b2-9350-5577077b1207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:01.064789 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-acca9ffa-3f40-49b2-9350-5577077b1207 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:01.066031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-acca9ffa-3f40-49b2-9350-5577077b1207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:01.074095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1357/2712] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:01.079344 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd7133-a0a3-40df-bc3d-cff62ec062df None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:01.089246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd7133-a0a3-40df-bc3d-cff62ec062df None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:01.094081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a327014-b037-476a-a1ed-82c892148922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:01.094671 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a327014-b037-476a-a1ed-82c892148922 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:01.094950 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a327014-b037-476a-a1ed-82c892148922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:01.095217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a327014-b037-476a-a1ed-82c892148922 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:01.095894 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a327014-b037-476a-a1ed-82c892148922 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:03:01.096228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a327014-b037-476a-a1ed-82c892148922 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:01.097849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a327014-b037-476a-a1ed-82c892148922 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:01.109714 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1356/2713] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 08:03:01.114355 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca9ffa-3f40-49b2-9350-5577077b1207 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:01.124394 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca9ffa-3f40-49b2-9350-5577077b1207 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:01.128904 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7901ef-4ae6-4254-a1c8-0693b5f05c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:01.129489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7901ef-4ae6-4254-a1c8-0693b5f05c3f admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:01.129762 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7901ef-4ae6-4254-a1c8-0693b5f05c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:01.130031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7901ef-4ae6-4254-a1c8-0693b5f05c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:01.130649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7901ef-4ae6-4254-a1c8-0693b5f05c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:01.132641 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7901ef-4ae6-4254-a1c8-0693b5f05c3f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:01.159777 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1358/2714] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:03:01 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:03:01.163841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a327014-b037-476a-a1ed-82c892148922 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:01.174596 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a327014-b037-476a-a1ed-82c892148922 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:01.231149 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddfaaa-5631-4889-8d51-7c595fd45b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:01.232333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddfaaa-5631-4889-8d51-7c595fd45b4a admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:01.232742 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddfaaa-5631-4889-8d51-7c595fd45b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:01.233018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddfaaa-5631-4889-8d51-7c595fd45b4a admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:01.233655 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ddfaaa-5631-4889-8d51-7c595fd45b4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:01.235530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ddfaaa-5631-4889-8d51-7c595fd45b4a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:01.257954 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1357/2715] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:03:01 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:03:01.262105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7901ef-4ae6-4254-a1c8-0693b5f05c3f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:01.275590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7901ef-4ae6-4254-a1c8-0693b5f05c3f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:01.280987 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882e9a8-5e36-45df-a2b8-ae53d85a5697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:01.281611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882e9a8-5e36-45df-a2b8-ae53d85a5697 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:01.281882 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882e9a8-5e36-45df-a2b8-ae53d85a5697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:01.282151 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882e9a8-5e36-45df-a2b8-ae53d85a5697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:01.282793 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882e9a8-5e36-45df-a2b8-ae53d85a5697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:01.284434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882e9a8-5e36-45df-a2b8-ae53d85a5697 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:01.295319 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1359/2716] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:01.299843 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddfaaa-5631-4889-8d51-7c595fd45b4a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:01.310343 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddfaaa-5631-4889-8d51-7c595fd45b4a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:01.315366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6313e0f-9599-4628-a014-1a7c59d3fc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:01.315961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6313e0f-9599-4628-a014-1a7c59d3fc91 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:01.316269 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6313e0f-9599-4628-a014-1a7c59d3fc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:01.316538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6313e0f-9599-4628-a014-1a7c59d3fc91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:01.317166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6313e0f-9599-4628-a014-1a7c59d3fc91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:01.318735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6313e0f-9599-4628-a014-1a7c59d3fc91 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:01.329797 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1358/2717] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:01.335484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882e9a8-5e36-45df-a2b8-ae53d85a5697 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:01.345851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882e9a8-5e36-45df-a2b8-ae53d85a5697 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:01.350542 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16e02f-d3b0-44a1-988d-f80a3ec6d139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:01.351195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16e02f-d3b0-44a1-988d-f80a3ec6d139 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:01.351506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16e02f-d3b0-44a1-988d-f80a3ec6d139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:01.351781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16e02f-d3b0-44a1-988d-f80a3ec6d139 admin admin] PATH_INFO: `/v3/projects/56274d4cbc914a89b6165ed840076b55/users/275848f9d76945649f1c0b454a498aff/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:01.367700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16e02f-d3b0-44a1-988d-f80a3ec6d139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56274d4cbc914a89b6165ed840076b55, user_id=275848f9d76945649f1c0b454a498aff, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:01.370469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16e02f-d3b0-44a1-988d-f80a3ec6d139 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:01.385138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1360/2718] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:03:01 2022] PUT /identity/v3/projects/56274d4cbc914a89b6165ed840076b55/users/275848f9d76945649f1c0b454a498aff/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:01.396358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51949182-d42f-401e-9f2a-9d2799eec0d7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:01.396638 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51949182-d42f-401e-9f2a-9d2799eec0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:01.396909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51949182-d42f-401e-9f2a-9d2799eec0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:01.422645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:01.426463 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-51949182-d42f-401e-9f2a-9d2799eec0d7 None None] MFA Rules not processed for user `275848f9d76945649f1c0b454a498aff`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:01.428737 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-51949182-d42f-401e-9f2a-9d2799eec0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:01.496856 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1359/2719] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:01.501848 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16e02f-d3b0-44a1-988d-f80a3ec6d139 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:01.576031 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16e02f-d3b0-44a1-988d-f80a3ec6d139 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:01.580436 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f951fd6-9b49-4a72-adb9-40dfe2547542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:01.581037 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f951fd6-9b49-4a72-adb9-40dfe2547542 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:01.581318 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f951fd6-9b49-4a72-adb9-40dfe2547542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:01.581590 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f951fd6-9b49-4a72-adb9-40dfe2547542 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:01.582209 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f951fd6-9b49-4a72-adb9-40dfe2547542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:01.584396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f951fd6-9b49-4a72-adb9-40dfe2547542 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:01.601923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:01.619823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1361/2720] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:03:01 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:03:01.624614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51949182-d42f-401e-9f2a-9d2799eec0d7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:01.636162 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51949182-d42f-401e-9f2a-9d2799eec0d7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:01.823341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45246cb3-1864-48b4-a76f-0364359ce366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:01.824004 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45246cb3-1864-48b4-a76f-0364359ce366 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:01.824382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45246cb3-1864-48b4-a76f-0364359ce366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:01.824683 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45246cb3-1864-48b4-a76f-0364359ce366 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:01.826157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45246cb3-1864-48b4-a76f-0364359ce366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:01.828421 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45246cb3-1864-48b4-a76f-0364359ce366 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:01.854132 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1360/2721] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:03:01 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:03:01.855308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f951fd6-9b49-4a72-adb9-40dfe2547542 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:01.865588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f951fd6-9b49-4a72-adb9-40dfe2547542 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:01.870536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19ac60-cc23-41c1-973d-7d19c430d7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:01.871154 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19ac60-cc23-41c1-973d-7d19c430d7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:01.871428 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19ac60-cc23-41c1-973d-7d19c430d7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:01.871696 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19ac60-cc23-41c1-973d-7d19c430d7f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:01.872372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19ac60-cc23-41c1-973d-7d19c430d7f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:01.873948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19ac60-cc23-41c1-973d-7d19c430d7f7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:01.887334 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1362/2722] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:01.889644 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45246cb3-1864-48b4-a76f-0364359ce366 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:01.900290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45246cb3-1864-48b4-a76f-0364359ce366 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:01.905915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3d1a9-23fe-4fb2-a7b9-b1c004254311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:01.906538 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3d1a9-23fe-4fb2-a7b9-b1c004254311 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:01.906814 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3d1a9-23fe-4fb2-a7b9-b1c004254311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:01.907078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3d1a9-23fe-4fb2-a7b9-b1c004254311 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:01.907708 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3d1a9-23fe-4fb2-a7b9-b1c004254311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:01.909386 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3d1a9-23fe-4fb2-a7b9-b1c004254311 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:01.920606 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1361/2723] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:01.925127 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19ac60-cc23-41c1-973d-7d19c430d7f7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:01.930929 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3d1a9-23fe-4fb2-a7b9-b1c004254311 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:01.932722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-73a3d1a9-23fe-4fb2-a7b9-b1c004254311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:01.937159 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19ac60-cc23-41c1-973d-7d19c430d7f7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:01.942651 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ceb057-723b-43a9-9d04-0ec7495bc614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:01.943376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ceb057-723b-43a9-9d04-0ec7495bc614 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:01.943730 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ceb057-723b-43a9-9d04-0ec7495bc614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:01.944239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ceb057-723b-43a9-9d04-0ec7495bc614 admin admin] PATH_INFO: `/v3/projects/3204e5bc098b4909b75b00738580a912/users/8938133f20a24656bb6f68f074a56497/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:01.962518 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ceb057-723b-43a9-9d04-0ec7495bc614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3204e5bc098b4909b75b00738580a912, user_id=8938133f20a24656bb6f68f074a56497, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:01.965263 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ceb057-723b-43a9-9d04-0ec7495bc614 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:01.979486 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1363/2724] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:03:01 2022] PUT /identity/v3/projects/3204e5bc098b4909b75b00738580a912/users/8938133f20a24656bb6f68f074a56497/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:01.990163 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815119a9-2ef7-49af-a8d5-5d4ccc38d12c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:01.990532 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815119a9-2ef7-49af-a8d5-5d4ccc38d12c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:01.990810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815119a9-2ef7-49af-a8d5-5d4ccc38d12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:02.003339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3d1a9-23fe-4fb2-a7b9-b1c004254311 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:02.020365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-815119a9-2ef7-49af-a8d5-5d4ccc38d12c None None] MFA Rules not processed for user `8938133f20a24656bb6f68f074a56497`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:02.021776 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-815119a9-2ef7-49af-a8d5-5d4ccc38d12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:02.037259 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:02.053971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bba457-b41f-4724-b838-ab507a866392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:02.054667 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bba457-b41f-4724-b838-ab507a866392 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:02.054952 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bba457-b41f-4724-b838-ab507a866392 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:02.055319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bba457-b41f-4724-b838-ab507a866392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:02.057327 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-89bba457-b41f-4724-b838-ab507a866392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:02.095682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1364/2725] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:03:02.100161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815119a9-2ef7-49af-a8d5-5d4ccc38d12c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:02.125798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bba457-b41f-4724-b838-ab507a866392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:02.129623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bba457-b41f-4724-b838-ab507a866392 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:02.145321 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1362/2726] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:03:02.175810 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815119a9-2ef7-49af-a8d5-5d4ccc38d12c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:02.181986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334388e2-4ab0-48f5-9e6f-48e3ab687393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:02.182659 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334388e2-4ab0-48f5-9e6f-48e3ab687393 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:02.183022 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334388e2-4ab0-48f5-9e6f-48e3ab687393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:02.183351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334388e2-4ab0-48f5-9e6f-48e3ab687393 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:02.184196 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334388e2-4ab0-48f5-9e6f-48e3ab687393 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:02.187576 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334388e2-4ab0-48f5-9e6f-48e3ab687393 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:02.223248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1365/2727] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:03:02 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:03:02.227872 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bba457-b41f-4724-b838-ab507a866392 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:02.239359 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bba457-b41f-4724-b838-ab507a866392 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:02.317201 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a3033-b8de-4ed3-94fd-2a3bcc267aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:02.317999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a3033-b8de-4ed3-94fd-2a3bcc267aaf admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:02.318507 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a3033-b8de-4ed3-94fd-2a3bcc267aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:02.318968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a3033-b8de-4ed3-94fd-2a3bcc267aaf admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:02.319918 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a3033-b8de-4ed3-94fd-2a3bcc267aaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:02.322378 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a3033-b8de-4ed3-94fd-2a3bcc267aaf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:02.349822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1363/2728] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:03:02 2022] POST /identity/v3/users => generated 295 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:03:02.353766 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334388e2-4ab0-48f5-9e6f-48e3ab687393 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:02.365304 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334388e2-4ab0-48f5-9e6f-48e3ab687393 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:02.369986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796eef1-7be8-431d-877a-f0dec83ec9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:02.370601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796eef1-7be8-431d-877a-f0dec83ec9fd admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:02.370884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796eef1-7be8-431d-877a-f0dec83ec9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:02.371159 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796eef1-7be8-431d-877a-f0dec83ec9fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:02.371802 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796eef1-7be8-431d-877a-f0dec83ec9fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:02.373547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796eef1-7be8-431d-877a-f0dec83ec9fd admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:02.385036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1366/2729] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:02.390711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a3033-b8de-4ed3-94fd-2a3bcc267aaf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:02.403305 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a3033-b8de-4ed3-94fd-2a3bcc267aaf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:02.427447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934cc63-da8e-4cf1-b387-50d7a3512fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:02.428156 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934cc63-da8e-4cf1-b387-50d7a3512fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:02.428438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934cc63-da8e-4cf1-b387-50d7a3512fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:02.428713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934cc63-da8e-4cf1-b387-50d7a3512fe5 admin admin] PATH_INFO: `/v3/projects/4a9fb4f7799b4e48b479c0c748f42df2/users/70da870fe96e4c1ba8b4d3edcc3db0ea/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:02.450180 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934cc63-da8e-4cf1-b387-50d7a3512fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9fb4f7799b4e48b479c0c748f42df2, user_id=70da870fe96e4c1ba8b4d3edcc3db0ea, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:02.453413 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934cc63-da8e-4cf1-b387-50d7a3512fe5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:02.472966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1364/2730] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:03:02 2022] PUT /identity/v3/projects/4a9fb4f7799b4e48b479c0c748f42df2/users/70da870fe96e4c1ba8b4d3edcc3db0ea/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:02.476832 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796eef1-7be8-431d-877a-f0dec83ec9fd admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:02.491631 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796eef1-7be8-431d-877a-f0dec83ec9fd admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:02.511613 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:02.553808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe60232-081c-469b-8b8e-3cbbc447a77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:02.554417 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe60232-081c-469b-8b8e-3cbbc447a77f admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:02.554698 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe60232-081c-469b-8b8e-3cbbc447a77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:02.554966 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe60232-081c-469b-8b8e-3cbbc447a77f admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:02.555598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe60232-081c-469b-8b8e-3cbbc447a77f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:02.557227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe60232-081c-469b-8b8e-3cbbc447a77f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:02.568024 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1367/2731] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:02.572425 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934cc63-da8e-4cf1-b387-50d7a3512fe5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:02.584317 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934cc63-da8e-4cf1-b387-50d7a3512fe5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:02.588920 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114489f-75a3-4fba-9f35-bac070a6cd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:02.589568 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114489f-75a3-4fba-9f35-bac070a6cd4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:02.589840 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114489f-75a3-4fba-9f35-bac070a6cd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:02.590109 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114489f-75a3-4fba-9f35-bac070a6cd4b admin admin] PATH_INFO: `/v3/projects/4a9fb4f7799b4e48b479c0c748f42df2/users/70da870fe96e4c1ba8b4d3edcc3db0ea/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:02.592654 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114489f-75a3-4fba-9f35-bac070a6cd4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9fb4f7799b4e48b479c0c748f42df2, user_id=70da870fe96e4c1ba8b4d3edcc3db0ea, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:02.595237 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114489f-75a3-4fba-9f35-bac070a6cd4b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:02.606118 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1365/2732] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:03:02 2022] PUT /identity/v3/projects/4a9fb4f7799b4e48b479c0c748f42df2/users/70da870fe96e4c1ba8b4d3edcc3db0ea/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:02.616143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e57ac-eac3-4bab-bbf8-e3ec985ba35b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:02.616401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e57ac-eac3-4bab-bbf8-e3ec985ba35b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:02.616704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e57ac-eac3-4bab-bbf8-e3ec985ba35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:02.646343 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-646e57ac-eac3-4bab-bbf8-e3ec985ba35b None None] MFA Rules not processed for user `70da870fe96e4c1ba8b4d3edcc3db0ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:02.647618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-646e57ac-eac3-4bab-bbf8-e3ec985ba35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:02.728662 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1368/2733] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:02.741083 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114489f-75a3-4fba-9f35-bac070a6cd4b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:02.742894 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1114489f-75a3-4fba-9f35-bac070a6cd4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:02.791468 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:02.804643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114489f-75a3-4fba-9f35-bac070a6cd4b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:02.808389 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57627c-5447-4654-b358-1346133fff62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:02.809017 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57627c-5447-4654-b358-1346133fff62 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:02.809293 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57627c-5447-4654-b358-1346133fff62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:02.810264 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57627c-5447-4654-b358-1346133fff62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:02.870942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57627c-5447-4654-b358-1346133fff62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:02.872545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57627c-5447-4654-b358-1346133fff62 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:02.882407 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1366/2734] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:03.073373 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e57ac-eac3-4bab-bbf8-e3ec985ba35b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:03.075312 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-646e57ac-eac3-4bab-bbf8-e3ec985ba35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:03.163875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e57ac-eac3-4bab-bbf8-e3ec985ba35b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:03.167409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5962500-ff46-4207-a835-85b21248b9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:03.168034 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5962500-ff46-4207-a835-85b21248b9a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:03.168343 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5962500-ff46-4207-a835-85b21248b9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:03.168614 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5962500-ff46-4207-a835-85b21248b9a9 admin admin] PATH_INFO: `/v3/users/3bf34b9f97cd499593dc157f00e075cd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:03.170193 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5962500-ff46-4207-a835-85b21248b9a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bf34b9f97cd499593dc157f00e075cd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:03.172323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5962500-ff46-4207-a835-85b21248b9a9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:03.251683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:03.254361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e5962500-ff46-4207-a835-85b21248b9a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bf34b9f97cd499593dc157f00e075cd'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:03.255307 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e5962500-ff46-4207-a835-85b21248b9a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bf34b9f97cd499593dc157f00e075cd'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:03.273392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e5962500-ff46-4207-a835-85b21248b9a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bf34b9f97cd499593dc157f00e075cd'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:03.274178 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e5962500-ff46-4207-a835-85b21248b9a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bf34b9f97cd499593dc157f00e075cd'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:03.283136 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e5962500-ff46-4207-a835-85b21248b9a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bf34b9f97cd499593dc157f00e075cd'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:03.298176 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1369/2735] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:03:03 2022] DELETE /identity/v3/users/3bf34b9f97cd499593dc157f00e075cd => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:03.302616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5589bf-898c-41b3-89a3-5a0e13318ec7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:03.303098 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5589bf-898c-41b3-89a3-5a0e13318ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:03.303214 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5589bf-898c-41b3-89a3-5a0e13318ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:03.323285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6f5589bf-898c-41b3-89a3-5a0e13318ec7 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:03.324522 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6f5589bf-898c-41b3-89a3-5a0e13318ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:03.390529 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1367/2736] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:03.400414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5962500-ff46-4207-a835-85b21248b9a9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:03.401821 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e5962500-ff46-4207-a835-85b21248b9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:03.464120 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5962500-ff46-4207-a835-85b21248b9a9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:03.468387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532756c-3037-47dc-8d2f-c7701a1a97ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:03.468991 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532756c-3037-47dc-8d2f-c7701a1a97ea service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:03.469266 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532756c-3037-47dc-8d2f-c7701a1a97ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:03.469535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532756c-3037-47dc-8d2f-c7701a1a97ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:03.482252 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532756c-3037-47dc-8d2f-c7701a1a97ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:03.484268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532756c-3037-47dc-8d2f-c7701a1a97ea service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:03.495416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1370/2737] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:03.928198 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5589bf-898c-41b3-89a3-5a0e13318ec7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:03.929792 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6f5589bf-898c-41b3-89a3-5a0e13318ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:03.942764 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5589bf-898c-41b3-89a3-5a0e13318ec7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:03.947627 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7c9be-9715-4213-9830-14a0f07bdc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:03.948121 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7c9be-9715-4213-9830-14a0f07bdc8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:03.948427 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7c9be-9715-4213-9830-14a0f07bdc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:03.948664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7c9be-9715-4213-9830-14a0f07bdc8f admin admin] PATH_INFO: `/v3/projects/8d677351e7cb4f70a4348858d5a51e5a` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:03.949793 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f7c9be-9715-4213-9830-14a0f07bdc8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d677351e7cb4f70a4348858d5a51e5a)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:03.952289 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f7c9be-9715-4213-9830-14a0f07bdc8f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:04.017426 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-91f7c9be-9715-4213-9830-14a0f07bdc8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d677351e7cb4f70a4348858d5a51e5a', 'request_id': 'req-91f7c9be-9715-4213-9830-14a0f07bdc8f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.023332 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-91f7c9be-9715-4213-9830-14a0f07bdc8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d677351e7cb4f70a4348858d5a51e5a', 'request_id': 'req-91f7c9be-9715-4213-9830-14a0f07bdc8f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.030246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-91f7c9be-9715-4213-9830-14a0f07bdc8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d677351e7cb4f70a4348858d5a51e5a', 'request_id': 'req-91f7c9be-9715-4213-9830-14a0f07bdc8f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.043370 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-91f7c9be-9715-4213-9830-14a0f07bdc8f admin admin] The token cache is being invalidate because project 8d677351e7cb4f70a4348858d5a51e5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:04.043738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-91f7c9be-9715-4213-9830-14a0f07bdc8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.045013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-91f7c9be-9715-4213-9830-14a0f07bdc8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.046545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1368/2738] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:03 2022] DELETE /identity/v3/projects/8d677351e7cb4f70a4348858d5a51e5a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:04.050711 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532756c-3037-47dc-8d2f-c7701a1a97ea service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:04.061411 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532756c-3037-47dc-8d2f-c7701a1a97ea service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:04.120278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ac90c-e5d5-45c5-9d11-6c73ae86df11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:04.120902 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ac90c-e5d5-45c5-9d11-6c73ae86df11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:04.121162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ac90c-e5d5-45c5-9d11-6c73ae86df11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:04.121430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ac90c-e5d5-45c5-9d11-6c73ae86df11 admin admin] PATH_INFO: `/v3/users/8f90574e18a04a17b515aa87e2e1ce48` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:04.123118 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ac90c-e5d5-45c5-9d11-6c73ae86df11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f90574e18a04a17b515aa87e2e1ce48)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:04.125087 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ac90c-e5d5-45c5-9d11-6c73ae86df11 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:04.174673 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ba5ac90c-e5d5-45c5-9d11-6c73ae86df11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f90574e18a04a17b515aa87e2e1ce48'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.175354 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ba5ac90c-e5d5-45c5-9d11-6c73ae86df11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f90574e18a04a17b515aa87e2e1ce48'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.188204 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ba5ac90c-e5d5-45c5-9d11-6c73ae86df11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f90574e18a04a17b515aa87e2e1ce48'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.188861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ba5ac90c-e5d5-45c5-9d11-6c73ae86df11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f90574e18a04a17b515aa87e2e1ce48'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.190111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:04.199127 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ba5ac90c-e5d5-45c5-9d11-6c73ae86df11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f90574e18a04a17b515aa87e2e1ce48'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.213009 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1371/2739] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:03:04 2022] DELETE /identity/v3/users/8f90574e18a04a17b515aa87e2e1ce48 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:04.488333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7c9be-9715-4213-9830-14a0f07bdc8f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:04.490417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-91f7c9be-9715-4213-9830-14a0f07bdc8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:04.522985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ac90c-e5d5-45c5-9d11-6c73ae86df11 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:04.524947 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ba5ac90c-e5d5-45c5-9d11-6c73ae86df11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:04.530149 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:04.576497 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7c9be-9715-4213-9830-14a0f07bdc8f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:04.619171 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ac90c-e5d5-45c5-9d11-6c73ae86df11 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:04.625752 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b939c0-aecf-42eb-a88a-ff2f7edab304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:04.626475 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b939c0-aecf-42eb-a88a-ff2f7edab304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:04.626825 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b939c0-aecf-42eb-a88a-ff2f7edab304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:04.627197 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b939c0-aecf-42eb-a88a-ff2f7edab304 admin admin] PATH_INFO: `/v3/projects/0f1967ce6f4f4824ab8a46befea004cd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:04.628664 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b939c0-aecf-42eb-a88a-ff2f7edab304 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f1967ce6f4f4824ab8a46befea004cd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:04.630656 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b939c0-aecf-42eb-a88a-ff2f7edab304 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:04.633186 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7434aaa-2cc9-4dcc-bd8b-661422a01781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:04.634112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7434aaa-2cc9-4dcc-bd8b-661422a01781 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:04.634385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7434aaa-2cc9-4dcc-bd8b-661422a01781 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:04.634653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7434aaa-2cc9-4dcc-bd8b-661422a01781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:04.636430 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b7434aaa-2cc9-4dcc-bd8b-661422a01781 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:04.686329 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-66b939c0-aecf-42eb-a88a-ff2f7edab304 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f1967ce6f4f4824ab8a46befea004cd', 'request_id': 'req-66b939c0-aecf-42eb-a88a-ff2f7edab304'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.693546 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-66b939c0-aecf-42eb-a88a-ff2f7edab304 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f1967ce6f4f4824ab8a46befea004cd', 'request_id': 'req-66b939c0-aecf-42eb-a88a-ff2f7edab304'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.701811 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-66b939c0-aecf-42eb-a88a-ff2f7edab304 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f1967ce6f4f4824ab8a46befea004cd', 'request_id': 'req-66b939c0-aecf-42eb-a88a-ff2f7edab304'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.706047 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7434aaa-2cc9-4dcc-bd8b-661422a01781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:04.708181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7434aaa-2cc9-4dcc-bd8b-661422a01781 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:04.715539 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-66b939c0-aecf-42eb-a88a-ff2f7edab304 admin admin] The token cache is being invalidate because project 0f1967ce6f4f4824ab8a46befea004cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:04.715997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-66b939c0-aecf-42eb-a88a-ff2f7edab304 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.716979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-66b939c0-aecf-42eb-a88a-ff2f7edab304 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.718518 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1372/2740] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:04 2022] DELETE /identity/v3/projects/0f1967ce6f4f4824ab8a46befea004cd => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:03:04.721488 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1369/2741] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:04.722248 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b939c0-aecf-42eb-a88a-ff2f7edab304 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:04.723535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-66b939c0-aecf-42eb-a88a-ff2f7edab304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:04.791477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b939c0-aecf-42eb-a88a-ff2f7edab304 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:04.796775 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049c559-498f-4077-8129-8db19dff3ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:04.797416 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049c559-498f-4077-8129-8db19dff3ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:04.797690 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049c559-498f-4077-8129-8db19dff3ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:04.797958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049c559-498f-4077-8129-8db19dff3ee7 admin admin] PATH_INFO: `/v3/users/9c349a3f1a3e4b6e900177f8e90353e3` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:04.799087 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049c559-498f-4077-8129-8db19dff3ee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c349a3f1a3e4b6e900177f8e90353e3)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:04.801049 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049c559-498f-4077-8129-8db19dff3ee7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:04.816398 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:04.855712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4049c559-498f-4077-8129-8db19dff3ee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c349a3f1a3e4b6e900177f8e90353e3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.857041 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4049c559-498f-4077-8129-8db19dff3ee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c349a3f1a3e4b6e900177f8e90353e3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.869340 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4049c559-498f-4077-8129-8db19dff3ee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c349a3f1a3e4b6e900177f8e90353e3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.870006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4049c559-498f-4077-8129-8db19dff3ee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c349a3f1a3e4b6e900177f8e90353e3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.877358 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4049c559-498f-4077-8129-8db19dff3ee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c349a3f1a3e4b6e900177f8e90353e3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:04.892959 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1373/2742] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:03:04 2022] DELETE /identity/v3/users/9c349a3f1a3e4b6e900177f8e90353e3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:03:05.362065 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7434aaa-2cc9-4dcc-bd8b-661422a01781 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:05.373733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049c559-498f-4077-8129-8db19dff3ee7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:05.375411 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4049c559-498f-4077-8129-8db19dff3ee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:05.391217 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7434aaa-2cc9-4dcc-bd8b-661422a01781 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:05.450733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049c559-498f-4077-8129-8db19dff3ee7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:05.456032 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721438b7-49ae-46a2-b2bd-c60028e00a4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:05.456913 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721438b7-49ae-46a2-b2bd-c60028e00a4a service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:05.457345 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721438b7-49ae-46a2-b2bd-c60028e00a4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:05.457879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721438b7-49ae-46a2-b2bd-c60028e00a4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:05.459851 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-721438b7-49ae-46a2-b2bd-c60028e00a4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:05.473971 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de07e3-f991-46f2-98d6-5134dae8148c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:05.474620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de07e3-f991-46f2-98d6-5134dae8148c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:05.474900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de07e3-f991-46f2-98d6-5134dae8148c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:05.475176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de07e3-f991-46f2-98d6-5134dae8148c admin admin] PATH_INFO: `/v3/projects/25864af38b7043718f67cde53c62ea1d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:05.476423 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de07e3-f991-46f2-98d6-5134dae8148c admin admin] RBAC: Authorizing `identity:delete_project(project_id=25864af38b7043718f67cde53c62ea1d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:05.478376 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de07e3-f991-46f2-98d6-5134dae8148c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:05.525526 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:05.538610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-13de07e3-f991-46f2-98d6-5134dae8148c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25864af38b7043718f67cde53c62ea1d', 'request_id': 'req-13de07e3-f991-46f2-98d6-5134dae8148c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:05.540452 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721438b7-49ae-46a2-b2bd-c60028e00a4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:05.542147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721438b7-49ae-46a2-b2bd-c60028e00a4a service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:05.545377 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-13de07e3-f991-46f2-98d6-5134dae8148c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25864af38b7043718f67cde53c62ea1d', 'request_id': 'req-13de07e3-f991-46f2-98d6-5134dae8148c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:05.551868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-13de07e3-f991-46f2-98d6-5134dae8148c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25864af38b7043718f67cde53c62ea1d', 'request_id': 'req-13de07e3-f991-46f2-98d6-5134dae8148c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:05.553294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1374/2743] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:05.573114 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-13de07e3-f991-46f2-98d6-5134dae8148c admin admin] The token cache is being invalidate because project 25864af38b7043718f67cde53c62ea1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:05.573480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-13de07e3-f991-46f2-98d6-5134dae8148c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:05.574339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-13de07e3-f991-46f2-98d6-5134dae8148c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:05.576725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1370/2744] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:05 2022] DELETE /identity/v3/projects/25864af38b7043718f67cde53c62ea1d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:03:05.582973 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435559a7-1e39-468b-8fe8-ce29a29524e3 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:05.583260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435559a7-1e39-468b-8fe8-ce29a29524e3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:05.583535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435559a7-1e39-468b-8fe8-ce29a29524e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:05.605518 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-435559a7-1e39-468b-8fe8-ce29a29524e3 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:05.606764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-435559a7-1e39-468b-8fe8-ce29a29524e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:05.655977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:05.685356 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1375/2745] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:03:05.694870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28885e28-d92a-4efc-b1b7-7961e12cb54c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:05.695155 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28885e28-d92a-4efc-b1b7-7961e12cb54c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:05.695428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28885e28-d92a-4efc-b1b7-7961e12cb54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:05.715479 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-28885e28-d92a-4efc-b1b7-7961e12cb54c None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:05.717246 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-28885e28-d92a-4efc-b1b7-7961e12cb54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:05.725131 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1371/2746] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:05.729427 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435559a7-1e39-468b-8fe8-ce29a29524e3 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:05.740691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435559a7-1e39-468b-8fe8-ce29a29524e3 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:05.745373 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e220ff-c584-4152-b795-dd87ceab057d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:05.746002 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e220ff-c584-4152-b795-dd87ceab057d admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:05.746280 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e220ff-c584-4152-b795-dd87ceab057d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:05.746550 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e220ff-c584-4152-b795-dd87ceab057d admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:05.747430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e220ff-c584-4152-b795-dd87ceab057d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:03:05.747833 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e220ff-c584-4152-b795-dd87ceab057d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:05.749490 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e220ff-c584-4152-b795-dd87ceab057d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:05.760958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1376/2747] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:03:05.765440 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28885e28-d92a-4efc-b1b7-7961e12cb54c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:05.774867 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28885e28-d92a-4efc-b1b7-7961e12cb54c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:05.780175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107fa29-ad6e-4614-8da1-c56fac933821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:05.780713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107fa29-ad6e-4614-8da1-c56fac933821 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:05.780992 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107fa29-ad6e-4614-8da1-c56fac933821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:05.781261 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107fa29-ad6e-4614-8da1-c56fac933821 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:05.781892 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107fa29-ad6e-4614-8da1-c56fac933821 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:05.783753 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107fa29-ad6e-4614-8da1-c56fac933821 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:05.811178 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1372/2748] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:03:05 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:03:05.817975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e220ff-c584-4152-b795-dd87ceab057d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:05.827934 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e220ff-c584-4152-b795-dd87ceab057d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:05.890495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db4e62-690c-47fa-93fd-2a1607f1da5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:05.891105 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db4e62-690c-47fa-93fd-2a1607f1da5b admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:05.891387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db4e62-690c-47fa-93fd-2a1607f1da5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:05.891670 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db4e62-690c-47fa-93fd-2a1607f1da5b admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:05.892337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db4e62-690c-47fa-93fd-2a1607f1da5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:05.894296 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db4e62-690c-47fa-93fd-2a1607f1da5b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:05.916675 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1377/2749] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:03:05 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:03:05.921533 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107fa29-ad6e-4614-8da1-c56fac933821 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:05.933105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107fa29-ad6e-4614-8da1-c56fac933821 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:05.938339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01c495-265e-4f35-928c-5686f6bd9da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:05.938931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01c495-265e-4f35-928c-5686f6bd9da2 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:05.939235 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01c495-265e-4f35-928c-5686f6bd9da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:05.939503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01c495-265e-4f35-928c-5686f6bd9da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:05.940219 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01c495-265e-4f35-928c-5686f6bd9da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:05.941811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01c495-265e-4f35-928c-5686f6bd9da2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:05.952535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1373/2750] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:05.956862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db4e62-690c-47fa-93fd-2a1607f1da5b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:05.967509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db4e62-690c-47fa-93fd-2a1607f1da5b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:05.973885 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b40c7-a8e7-42a3-93b7-084adeade10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:05.974493 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b40c7-a8e7-42a3-93b7-084adeade10a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:05.974820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b40c7-a8e7-42a3-93b7-084adeade10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:05.975503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b40c7-a8e7-42a3-93b7-084adeade10a admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:05.976186 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b40c7-a8e7-42a3-93b7-084adeade10a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:05.978124 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b40c7-a8e7-42a3-93b7-084adeade10a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:05.998756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1378/2751] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:06.000216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01c495-265e-4f35-928c-5686f6bd9da2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:06.011456 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01c495-265e-4f35-928c-5686f6bd9da2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:06.017052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67823da-3732-45c2-b03e-cb7e756f3dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:06.017705 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67823da-3732-45c2-b03e-cb7e756f3dc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:06.017985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67823da-3732-45c2-b03e-cb7e756f3dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:06.018255 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67823da-3732-45c2-b03e-cb7e756f3dc8 admin admin] PATH_INFO: `/v3/projects/f2376e070e2a4ee19033a277c535c95f/users/507fccc508664f81a6e4fd639302343d/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:06.038254 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67823da-3732-45c2-b03e-cb7e756f3dc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2376e070e2a4ee19033a277c535c95f, user_id=507fccc508664f81a6e4fd639302343d, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:06.041185 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67823da-3732-45c2-b03e-cb7e756f3dc8 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:06.063367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1374/2752] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:03:05 2022] PUT /identity/v3/projects/f2376e070e2a4ee19033a277c535c95f/users/507fccc508664f81a6e4fd639302343d/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:06.069672 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b6ad7-20a9-4caf-96d0-c6b246110af0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:06.070048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b6ad7-20a9-4caf-96d0-c6b246110af0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:06.070409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b6ad7-20a9-4caf-96d0-c6b246110af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:06.090041 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e50b6ad7-20a9-4caf-96d0-c6b246110af0 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:06.091437 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e50b6ad7-20a9-4caf-96d0-c6b246110af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:06.167623 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1379/2753] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:06.174670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67823da-3732-45c2-b03e-cb7e756f3dc8 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:06.176394 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b67823da-3732-45c2-b03e-cb7e756f3dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:06.247695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67823da-3732-45c2-b03e-cb7e756f3dc8 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:06.251523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe28491-4c12-43c1-9372-9d792c9adb27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:06.252156 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe28491-4c12-43c1-9372-9d792c9adb27 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:06.252620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe28491-4c12-43c1-9372-9d792c9adb27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:06.252965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe28491-4c12-43c1-9372-9d792c9adb27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:06.263489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe28491-4c12-43c1-9372-9d792c9adb27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:06.265079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe28491-4c12-43c1-9372-9d792c9adb27 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:06.275142 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1375/2754] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:07.074492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca812f70-5bbd-4cae-a451-0f40978d5ea6 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:07.075061 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca812f70-5bbd-4cae-a451-0f40978d5ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:07.075376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca812f70-5bbd-4cae-a451-0f40978d5ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:07.112528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ca812f70-5bbd-4cae-a451-0f40978d5ea6 None None] MFA Rules not processed for user `507fccc508664f81a6e4fd639302343d`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:07.113778 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ca812f70-5bbd-4cae-a451-0f40978d5ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:07.159148 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:07.196360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1380/2755] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:03:07.197675 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe28491-4c12-43c1-9372-9d792c9adb27 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:07.207228 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe28491-4c12-43c1-9372-9d792c9adb27 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:07.211724 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d1046-fdf6-4db0-b2f4-b0fc82f99c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:07.212339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d1046-fdf6-4db0-b2f4-b0fc82f99c8c admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:07.212610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d1046-fdf6-4db0-b2f4-b0fc82f99c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:07.212877 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d1046-fdf6-4db0-b2f4-b0fc82f99c8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:07.213494 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d1046-fdf6-4db0-b2f4-b0fc82f99c8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:07.215437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d1046-fdf6-4db0-b2f4-b0fc82f99c8c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:07.247399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1376/2756] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:03:07 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:03:07.248596 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca812f70-5bbd-4cae-a451-0f40978d5ea6 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:07.259411 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca812f70-5bbd-4cae-a451-0f40978d5ea6 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:07.328389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075346a6-fcc3-40fc-b79b-303e43e2c06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:07.329017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075346a6-fcc3-40fc-b79b-303e43e2c06f admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:07.329292 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075346a6-fcc3-40fc-b79b-303e43e2c06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:07.329558 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075346a6-fcc3-40fc-b79b-303e43e2c06f admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:07.330167 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075346a6-fcc3-40fc-b79b-303e43e2c06f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:07.332060 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075346a6-fcc3-40fc-b79b-303e43e2c06f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:07.356107 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1381/2757] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:03:07 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:03:07.360337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d1046-fdf6-4db0-b2f4-b0fc82f99c8c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:07.372070 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d1046-fdf6-4db0-b2f4-b0fc82f99c8c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:07.383443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a079e8-e5b7-4afd-8e6a-ac11b039bad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:07.384132 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a079e8-e5b7-4afd-8e6a-ac11b039bad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:07.384526 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a079e8-e5b7-4afd-8e6a-ac11b039bad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:07.385169 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a079e8-e5b7-4afd-8e6a-ac11b039bad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:07.388402 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a079e8-e5b7-4afd-8e6a-ac11b039bad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:07.393878 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a079e8-e5b7-4afd-8e6a-ac11b039bad1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:07.411180 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1377/2758] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:07.415609 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075346a6-fcc3-40fc-b79b-303e43e2c06f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:07.427045 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075346a6-fcc3-40fc-b79b-303e43e2c06f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:07.433026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441134c-06ae-4dc9-877a-4cf931d6caf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:07.433798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441134c-06ae-4dc9-877a-4cf931d6caf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:07.434076 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441134c-06ae-4dc9-877a-4cf931d6caf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:07.434440 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441134c-06ae-4dc9-877a-4cf931d6caf1 admin admin] PATH_INFO: `/v3/projects/4bd4cc8a886b4ed1a448cc9b08953d38/users/1f5b947a8aa34d6992d9cafa4af25137/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:07.450614 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441134c-06ae-4dc9-877a-4cf931d6caf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd4cc8a886b4ed1a448cc9b08953d38, user_id=1f5b947a8aa34d6992d9cafa4af25137, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:07.453298 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441134c-06ae-4dc9-877a-4cf931d6caf1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:07.467135 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1382/2759] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:03:07 2022] PUT /identity/v3/projects/4bd4cc8a886b4ed1a448cc9b08953d38/users/1f5b947a8aa34d6992d9cafa4af25137/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:03:07.471047 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a079e8-e5b7-4afd-8e6a-ac11b039bad1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:07.481374 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a079e8-e5b7-4afd-8e6a-ac11b039bad1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:07.508107 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:07.548040 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03509b1f-0929-4777-b081-ab7c25fac46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:07.548687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03509b1f-0929-4777-b081-ab7c25fac46b admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:07.548965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03509b1f-0929-4777-b081-ab7c25fac46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:07.549235 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03509b1f-0929-4777-b081-ab7c25fac46b admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:07.549871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03509b1f-0929-4777-b081-ab7c25fac46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:07.551455 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03509b1f-0929-4777-b081-ab7c25fac46b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:07.563171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1378/2760] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:07.566932 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441134c-06ae-4dc9-877a-4cf931d6caf1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:07.579064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441134c-06ae-4dc9-877a-4cf931d6caf1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:07.584007 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b65490-c29b-4718-91a2-ae977d9d5dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:07.584734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b65490-c29b-4718-91a2-ae977d9d5dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:07.585058 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b65490-c29b-4718-91a2-ae977d9d5dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:07.585280 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b65490-c29b-4718-91a2-ae977d9d5dc6 admin admin] PATH_INFO: `/v3/projects/4bd4cc8a886b4ed1a448cc9b08953d38/users/1f5b947a8aa34d6992d9cafa4af25137/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:07.587283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b65490-c29b-4718-91a2-ae977d9d5dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd4cc8a886b4ed1a448cc9b08953d38, user_id=1f5b947a8aa34d6992d9cafa4af25137, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:07.602507 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b65490-c29b-4718-91a2-ae977d9d5dc6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:07.623728 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1383/2761] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:03:07 2022] PUT /identity/v3/projects/4bd4cc8a886b4ed1a448cc9b08953d38/users/1f5b947a8aa34d6992d9cafa4af25137/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:08.431620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc2c17-4181-43e1-b330-41235ef91b2b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:08.432100 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc2c17-4181-43e1-b330-41235ef91b2b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:08.432325 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc2c17-4181-43e1-b330-41235ef91b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:08.467069 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8acc2c17-4181-43e1-b330-41235ef91b2b None None] MFA Rules not processed for user `1f5b947a8aa34d6992d9cafa4af25137`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:08.468401 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8acc2c17-4181-43e1-b330-41235ef91b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:08.579597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1379/2762] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:08.596584 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b65490-c29b-4718-91a2-ae977d9d5dc6 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:08.598540 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-87b65490-c29b-4718-91a2-ae977d9d5dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:08.700335 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b65490-c29b-4718-91a2-ae977d9d5dc6 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:08.705220 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae60ea1-dd97-45bd-bac0-d71750d74b0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:08.705953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae60ea1-dd97-45bd-bac0-d71750d74b0e service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:08.708407 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae60ea1-dd97-45bd-bac0-d71750d74b0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:08.708782 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae60ea1-dd97-45bd-bac0-d71750d74b0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:08.766641 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:08.789896 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae60ea1-dd97-45bd-bac0-d71750d74b0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:08.791522 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae60ea1-dd97-45bd-bac0-d71750d74b0e service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:08.803767 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1384/2763] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:09.692049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc2c17-4181-43e1-b330-41235ef91b2b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:09.693498 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8acc2c17-4181-43e1-b330-41235ef91b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:09.756771 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc2c17-4181-43e1-b330-41235ef91b2b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:09.761370 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc707dc-39d0-4f43-80a2-20e49126b409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:09.761989 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc707dc-39d0-4f43-80a2-20e49126b409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:09.762265 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc707dc-39d0-4f43-80a2-20e49126b409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:09.762572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc707dc-39d0-4f43-80a2-20e49126b409 admin admin] PATH_INFO: `/v3/users/275848f9d76945649f1c0b454a498aff` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:09.763761 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc707dc-39d0-4f43-80a2-20e49126b409 admin admin] RBAC: Authorizing `identity:delete_user(user_id=275848f9d76945649f1c0b454a498aff)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:09.765676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc707dc-39d0-4f43-80a2-20e49126b409 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:09.821292 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ccc707dc-39d0-4f43-80a2-20e49126b409 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '275848f9d76945649f1c0b454a498aff'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:09.829337 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:09.835233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ccc707dc-39d0-4f43-80a2-20e49126b409 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '275848f9d76945649f1c0b454a498aff'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:09.843946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ccc707dc-39d0-4f43-80a2-20e49126b409 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '275848f9d76945649f1c0b454a498aff'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:09.844597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ccc707dc-39d0-4f43-80a2-20e49126b409 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '275848f9d76945649f1c0b454a498aff'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:09.854080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ccc707dc-39d0-4f43-80a2-20e49126b409 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '275848f9d76945649f1c0b454a498aff'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:09.865140 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1380/2764] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:03:09 2022] DELETE /identity/v3/users/275848f9d76945649f1c0b454a498aff => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:09.875818 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57cce-77b9-4c12-bbaa-cd03fecacdf6 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:09.876439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57cce-77b9-4c12-bbaa-cd03fecacdf6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:09.876439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57cce-77b9-4c12-bbaa-cd03fecacdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:09.899071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-95a57cce-77b9-4c12-bbaa-cd03fecacdf6 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:09.900418 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-95a57cce-77b9-4c12-bbaa-cd03fecacdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:09.970106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1385/2765] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:09.982278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc707dc-39d0-4f43-80a2-20e49126b409 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:09.983937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ccc707dc-39d0-4f43-80a2-20e49126b409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:10.072207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc707dc-39d0-4f43-80a2-20e49126b409 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:10.076226 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8d2f4-c531-40ba-b492-1d89a0ab0939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:10.077368 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8d2f4-c531-40ba-b492-1d89a0ab0939 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:10.077610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8d2f4-c531-40ba-b492-1d89a0ab0939 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:10.077858 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8d2f4-c531-40ba-b492-1d89a0ab0939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:10.090288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8d2f4-c531-40ba-b492-1d89a0ab0939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:10.092341 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8d2f4-c531-40ba-b492-1d89a0ab0939 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:10.095617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a57cce-77b9-4c12-bbaa-cd03fecacdf6 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:10.097337 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-95a57cce-77b9-4c12-bbaa-cd03fecacdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:10.103121 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1381/2766] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:10.112941 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a57cce-77b9-4c12-bbaa-cd03fecacdf6 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:10.119647 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5fb37-34bf-4513-bf2d-285ca554b041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:10.120581 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5fb37-34bf-4513-bf2d-285ca554b041 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:10.120955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5fb37-34bf-4513-bf2d-285ca554b041 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:10.121495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5fb37-34bf-4513-bf2d-285ca554b041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:10.124067 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-23e5fb37-34bf-4513-bf2d-285ca554b041 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:10.142872 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e5fb37-34bf-4513-bf2d-285ca554b041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:10.145168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e5fb37-34bf-4513-bf2d-285ca554b041 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:10.159935 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1386/2767] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:03:10.212102 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8d2f4-c531-40ba-b492-1d89a0ab0939 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:10.213613 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1cd8d2f4-c531-40ba-b492-1d89a0ab0939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:10.299641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8d2f4-c531-40ba-b492-1d89a0ab0939 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:10.312273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b5e6a-bd64-4dcd-8763-d8ffb65bb03c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:10.313118 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b5e6a-bd64-4dcd-8763-d8ffb65bb03c service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:10.314036 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b5e6a-bd64-4dcd-8763-d8ffb65bb03c service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:10.314632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b5e6a-bd64-4dcd-8763-d8ffb65bb03c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:10.317409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-720b5e6a-bd64-4dcd-8763-d8ffb65bb03c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:10.413290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b5e6a-bd64-4dcd-8763-d8ffb65bb03c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:10.414902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b5e6a-bd64-4dcd-8763-d8ffb65bb03c service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:10.430165 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1382/2768] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:10.483198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5fb37-34bf-4513-bf2d-285ca554b041 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:10.485024 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-23e5fb37-34bf-4513-bf2d-285ca554b041 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:10.512055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5fb37-34bf-4513-bf2d-285ca554b041 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:10.520859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0f4ae-179f-457c-99ce-28db93c2aa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:10.521584 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0f4ae-179f-457c-99ce-28db93c2aa77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:10.521865 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0f4ae-179f-457c-99ce-28db93c2aa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:10.522172 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0f4ae-179f-457c-99ce-28db93c2aa77 admin admin] PATH_INFO: `/v3/projects/56274d4cbc914a89b6165ed840076b55` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:10.523574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0f4ae-179f-457c-99ce-28db93c2aa77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56274d4cbc914a89b6165ed840076b55)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:10.525553 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0f4ae-179f-457c-99ce-28db93c2aa77 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:10.591169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a9a0f4ae-179f-457c-99ce-28db93c2aa77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56274d4cbc914a89b6165ed840076b55', 'request_id': 'req-a9a0f4ae-179f-457c-99ce-28db93c2aa77'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:10.598188 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a9a0f4ae-179f-457c-99ce-28db93c2aa77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56274d4cbc914a89b6165ed840076b55', 'request_id': 'req-a9a0f4ae-179f-457c-99ce-28db93c2aa77'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:10.605801 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a9a0f4ae-179f-457c-99ce-28db93c2aa77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56274d4cbc914a89b6165ed840076b55', 'request_id': 'req-a9a0f4ae-179f-457c-99ce-28db93c2aa77'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:10.625288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a9a0f4ae-179f-457c-99ce-28db93c2aa77 admin admin] The token cache is being invalidate because project 56274d4cbc914a89b6165ed840076b55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:10.625838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a9a0f4ae-179f-457c-99ce-28db93c2aa77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:10.627149 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a9a0f4ae-179f-457c-99ce-28db93c2aa77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:10.629775 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1387/2769] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:10 2022] DELETE /identity/v3/projects/56274d4cbc914a89b6165ed840076b55 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:10.633752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b5e6a-bd64-4dcd-8763-d8ffb65bb03c service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:10.647672 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b5e6a-bd64-4dcd-8763-d8ffb65bb03c service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:10.678205 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:10.743490 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4675ecc-667f-4c7b-83ee-c408fe880f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:10.744177 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4675ecc-667f-4c7b-83ee-c408fe880f3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:10.744960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4675ecc-667f-4c7b-83ee-c408fe880f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:10.744960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4675ecc-667f-4c7b-83ee-c408fe880f3a admin admin] PATH_INFO: `/v3/users/8938133f20a24656bb6f68f074a56497` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:10.747396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4675ecc-667f-4c7b-83ee-c408fe880f3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8938133f20a24656bb6f68f074a56497)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:10.749900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4675ecc-667f-4c7b-83ee-c408fe880f3a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:10.828506 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e4675ecc-667f-4c7b-83ee-c408fe880f3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8938133f20a24656bb6f68f074a56497'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:10.845958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e4675ecc-667f-4c7b-83ee-c408fe880f3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8938133f20a24656bb6f68f074a56497'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:10.855277 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e4675ecc-667f-4c7b-83ee-c408fe880f3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8938133f20a24656bb6f68f074a56497'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:10.855922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e4675ecc-667f-4c7b-83ee-c408fe880f3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8938133f20a24656bb6f68f074a56497'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:10.864440 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e4675ecc-667f-4c7b-83ee-c408fe880f3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8938133f20a24656bb6f68f074a56497'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:10.872065 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1383/2770] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:03:10 2022] DELETE /identity/v3/users/8938133f20a24656bb6f68f074a56497 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:03:11.316015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0f4ae-179f-457c-99ce-28db93c2aa77 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:11.318660 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a9a0f4ae-179f-457c-99ce-28db93c2aa77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:11.333399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0f4ae-179f-457c-99ce-28db93c2aa77 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:11.405580 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:11.413565 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc08ff2-5863-4df7-a404-6ff26c4a585d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:11.414313 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc08ff2-5863-4df7-a404-6ff26c4a585d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:11.415134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc08ff2-5863-4df7-a404-6ff26c4a585d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:11.415134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc08ff2-5863-4df7-a404-6ff26c4a585d admin admin] PATH_INFO: `/v3/projects/3204e5bc098b4909b75b00738580a912` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:11.416213 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc08ff2-5863-4df7-a404-6ff26c4a585d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3204e5bc098b4909b75b00738580a912)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:11.418281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc08ff2-5863-4df7-a404-6ff26c4a585d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:11.477826 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ffc08ff2-5863-4df7-a404-6ff26c4a585d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3204e5bc098b4909b75b00738580a912', 'request_id': 'req-ffc08ff2-5863-4df7-a404-6ff26c4a585d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:11.485357 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ffc08ff2-5863-4df7-a404-6ff26c4a585d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3204e5bc098b4909b75b00738580a912', 'request_id': 'req-ffc08ff2-5863-4df7-a404-6ff26c4a585d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:11.491764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ffc08ff2-5863-4df7-a404-6ff26c4a585d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3204e5bc098b4909b75b00738580a912', 'request_id': 'req-ffc08ff2-5863-4df7-a404-6ff26c4a585d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:11.508762 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ffc08ff2-5863-4df7-a404-6ff26c4a585d admin admin] The token cache is being invalidate because project 3204e5bc098b4909b75b00738580a912 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:11.509050 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ffc08ff2-5863-4df7-a404-6ff26c4a585d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:11.509963 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ffc08ff2-5863-4df7-a404-6ff26c4a585d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:11.511288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1388/2771] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:11 2022] DELETE /identity/v3/projects/3204e5bc098b4909b75b00738580a912 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:11.516112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4675ecc-667f-4c7b-83ee-c408fe880f3a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:11.517830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e4675ecc-667f-4c7b-83ee-c408fe880f3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:11.533417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4675ecc-667f-4c7b-83ee-c408fe880f3a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:11.596941 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28469a3d-627f-41b6-8e88-111f07f756b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:11.597572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28469a3d-627f-41b6-8e88-111f07f756b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:11.597849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28469a3d-627f-41b6-8e88-111f07f756b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:11.598120 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28469a3d-627f-41b6-8e88-111f07f756b4 admin admin] PATH_INFO: `/v3/users/70da870fe96e4c1ba8b4d3edcc3db0ea` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:11.599233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28469a3d-627f-41b6-8e88-111f07f756b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70da870fe96e4c1ba8b4d3edcc3db0ea)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:11.601159 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28469a3d-627f-41b6-8e88-111f07f756b4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:11.647586 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:11.650315 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-28469a3d-627f-41b6-8e88-111f07f756b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70da870fe96e4c1ba8b4d3edcc3db0ea'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:11.665962 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-28469a3d-627f-41b6-8e88-111f07f756b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70da870fe96e4c1ba8b4d3edcc3db0ea'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:11.673288 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-28469a3d-627f-41b6-8e88-111f07f756b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70da870fe96e4c1ba8b4d3edcc3db0ea'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:11.673904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-28469a3d-627f-41b6-8e88-111f07f756b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70da870fe96e4c1ba8b4d3edcc3db0ea'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:11.680989 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-28469a3d-627f-41b6-8e88-111f07f756b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70da870fe96e4c1ba8b4d3edcc3db0ea'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:11.688051 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1384/2772] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:03:11 2022] DELETE /identity/v3/users/70da870fe96e4c1ba8b4d3edcc3db0ea => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:11.986869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc08ff2-5863-4df7-a404-6ff26c4a585d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:11.988520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ffc08ff2-5863-4df7-a404-6ff26c4a585d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:12.004974 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc08ff2-5863-4df7-a404-6ff26c4a585d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:12.083479 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c4b3e-9725-40a2-bd89-72a9777054ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:12.084138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c4b3e-9725-40a2-bd89-72a9777054ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:12.084419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c4b3e-9725-40a2-bd89-72a9777054ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:12.084687 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c4b3e-9725-40a2-bd89-72a9777054ba admin admin] PATH_INFO: `/v3/projects/4a9fb4f7799b4e48b479c0c748f42df2` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:12.086032 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c4b3e-9725-40a2-bd89-72a9777054ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a9fb4f7799b4e48b479c0c748f42df2)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:12.088878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c4b3e-9725-40a2-bd89-72a9777054ba admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:12.186966 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-df9c4b3e-9725-40a2-bd89-72a9777054ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a9fb4f7799b4e48b479c0c748f42df2', 'request_id': 'req-df9c4b3e-9725-40a2-bd89-72a9777054ba'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:12.207640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-df9c4b3e-9725-40a2-bd89-72a9777054ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a9fb4f7799b4e48b479c0c748f42df2', 'request_id': 'req-df9c4b3e-9725-40a2-bd89-72a9777054ba'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:12.215442 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-df9c4b3e-9725-40a2-bd89-72a9777054ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a9fb4f7799b4e48b479c0c748f42df2', 'request_id': 'req-df9c4b3e-9725-40a2-bd89-72a9777054ba'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:12.233013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-df9c4b3e-9725-40a2-bd89-72a9777054ba admin admin] The token cache is being invalidate because project 4a9fb4f7799b4e48b479c0c748f42df2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:12.233239 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-df9c4b3e-9725-40a2-bd89-72a9777054ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:12.233893 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-df9c4b3e-9725-40a2-bd89-72a9777054ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:12.238681 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1389/2773] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:11 2022] DELETE /identity/v3/projects/4a9fb4f7799b4e48b479c0c748f42df2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:12.240489 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b4f62-f241-429b-8954-3b33fa77d2ee None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:12.240959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b4f62-f241-429b-8954-3b33fa77d2ee None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:12.241086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b4f62-f241-429b-8954-3b33fa77d2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:12.262735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-579b4f62-f241-429b-8954-3b33fa77d2ee None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:12.264141 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-579b4f62-f241-429b-8954-3b33fa77d2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:12.331082 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1385/2774] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:12.340232 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b653fca-512c-4f77-b20d-2d4526f601ef None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:12.341986 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b653fca-512c-4f77-b20d-2d4526f601ef None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:12.342285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b653fca-512c-4f77-b20d-2d4526f601ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:12.365222 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7b653fca-512c-4f77-b20d-2d4526f601ef None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:12.366844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7b653fca-512c-4f77-b20d-2d4526f601ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:12.376719 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1390/2775] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:12.380927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b4f62-f241-429b-8954-3b33fa77d2ee None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:12.391608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b4f62-f241-429b-8954-3b33fa77d2ee None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:12.396831 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b77749-6d6d-4e8a-a850-2e77f06cc71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:12.397703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b77749-6d6d-4e8a-a850-2e77f06cc71a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:12.398174 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b77749-6d6d-4e8a-a850-2e77f06cc71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:12.398672 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b77749-6d6d-4e8a-a850-2e77f06cc71a admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:12.399519 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b77749-6d6d-4e8a-a850-2e77f06cc71a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:03:12.399892 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b77749-6d6d-4e8a-a850-2e77f06cc71a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:12.401561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b77749-6d6d-4e8a-a850-2e77f06cc71a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:12.415576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1386/2776] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:03:12.419740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b653fca-512c-4f77-b20d-2d4526f601ef None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:12.432665 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b653fca-512c-4f77-b20d-2d4526f601ef None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:12.438442 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511b89c-70af-4c73-814b-44863a691e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:12.439045 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511b89c-70af-4c73-814b-44863a691e2f admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:12.439572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511b89c-70af-4c73-814b-44863a691e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:12.439866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511b89c-70af-4c73-814b-44863a691e2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:12.440735 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1511b89c-70af-4c73-814b-44863a691e2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:12.442919 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1511b89c-70af-4c73-814b-44863a691e2f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:12.450985 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:12.615234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1391/2777] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:03:12 2022] POST /identity/v3/projects => generated 369 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:03:12.616136 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b77749-6d6d-4e8a-a850-2e77f06cc71a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:12.626759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b77749-6d6d-4e8a-a850-2e77f06cc71a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:12.692796 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f3dae-fc98-4185-bda3-5f5b162aae80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:12.693351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f3dae-fc98-4185-bda3-5f5b162aae80 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:12.693641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f3dae-fc98-4185-bda3-5f5b162aae80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:12.693904 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f3dae-fc98-4185-bda3-5f5b162aae80 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:12.696166 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f3dae-fc98-4185-bda3-5f5b162aae80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:12.696868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f3dae-fc98-4185-bda3-5f5b162aae80 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:12.720701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1387/2778] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:03:12 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:03:12.724509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511b89c-70af-4c73-814b-44863a691e2f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:12.735731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511b89c-70af-4c73-814b-44863a691e2f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:12.740848 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57000936-23a0-4c5c-ac90-5ae54310967e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:12.741536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57000936-23a0-4c5c-ac90-5ae54310967e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:12.741880 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57000936-23a0-4c5c-ac90-5ae54310967e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:12.742346 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57000936-23a0-4c5c-ac90-5ae54310967e admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:12.743240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57000936-23a0-4c5c-ac90-5ae54310967e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:12.744908 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57000936-23a0-4c5c-ac90-5ae54310967e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:12.756989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1392/2779] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:12.760932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f3dae-fc98-4185-bda3-5f5b162aae80 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:12.773617 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f3dae-fc98-4185-bda3-5f5b162aae80 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:12.779608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f3746-bcd4-4c0b-8da6-f59d38a13f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:12.780432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f3746-bcd4-4c0b-8da6-f59d38a13f85 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:12.780871 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f3746-bcd4-4c0b-8da6-f59d38a13f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:12.781303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f3746-bcd4-4c0b-8da6-f59d38a13f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:12.782099 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f3746-bcd4-4c0b-8da6-f59d38a13f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:12.784144 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f3746-bcd4-4c0b-8da6-f59d38a13f85 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:12.797406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1388/2780] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:12.801652 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57000936-23a0-4c5c-ac90-5ae54310967e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:12.813106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57000936-23a0-4c5c-ac90-5ae54310967e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:12.818406 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f2661-09c1-4c90-8c30-b78e475ae5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:12.819086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f2661-09c1-4c90-8c30-b78e475ae5b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:12.819364 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f2661-09c1-4c90-8c30-b78e475ae5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:12.819634 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f2661-09c1-4c90-8c30-b78e475ae5b7 admin admin] PATH_INFO: `/v3/projects/aa567ac48691480c908f6c69b0c8dfdb/users/feb497c6a7574866b0c6c4196a5500b6/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:12.848189 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f2661-09c1-4c90-8c30-b78e475ae5b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa567ac48691480c908f6c69b0c8dfdb, user_id=feb497c6a7574866b0c6c4196a5500b6, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:12.850865 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f2661-09c1-4c90-8c30-b78e475ae5b7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:12.866364 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1393/2781] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:03:12 2022] PUT /identity/v3/projects/aa567ac48691480c908f6c69b0c8dfdb/users/feb497c6a7574866b0c6c4196a5500b6/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:12.877299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e023cb-7ba4-46f7-82f0-d0c6c82db9d0 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:12.877653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e023cb-7ba4-46f7-82f0-d0c6c82db9d0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:12.877931 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e023cb-7ba4-46f7-82f0-d0c6c82db9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:12.907396 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c9e023cb-7ba4-46f7-82f0-d0c6c82db9d0 None None] MFA Rules not processed for user `feb497c6a7574866b0c6c4196a5500b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:12.908751 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c9e023cb-7ba4-46f7-82f0-d0c6c82db9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:12.921431 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:12.974622 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1389/2782] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:12.988161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f2661-09c1-4c90-8c30-b78e475ae5b7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:12.990559 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-128f2661-09c1-4c90-8c30-b78e475ae5b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:13.048227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f2661-09c1-4c90-8c30-b78e475ae5b7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:13.051896 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef349c9-9620-4530-9c0b-6b9e200e9761 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b0652c89c9945e2866512c9da4793ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:13.052593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef349c9-9620-4530-9c0b-6b9e200e9761 service glance] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:13.052857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef349c9-9620-4530-9c0b-6b9e200e9761 service glance] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:13.053238 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef349c9-9620-4530-9c0b-6b9e200e9761 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:13.064570 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef349c9-9620-4530-9c0b-6b9e200e9761 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:13.066003 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef349c9-9620-4530-9c0b-6b9e200e9761 service glance] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:13.075938 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1394/2783] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:15.080433 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e63912-f682-4b58-9b57-d2e624694df3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:15.080977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e63912-f682-4b58-9b57-d2e624694df3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:15.080977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e63912-f682-4b58-9b57-d2e624694df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:15.110028 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b0e63912-f682-4b58-9b57-d2e624694df3 None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:15.111231 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b0e63912-f682-4b58-9b57-d2e624694df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:15.213380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1390/2784] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Sep 7 08:03:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:15.264459 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef349c9-9620-4530-9c0b-6b9e200e9761 service glance] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:15.266541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7ef349c9-9620-4530-9c0b-6b9e200e9761 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:15.460279 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef349c9-9620-4530-9c0b-6b9e200e9761 service glance] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:15.467205 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c411332-26fb-4f95-921d-55fa469da370 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:15.468071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c411332-26fb-4f95-921d-55fa469da370 service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:15.468572 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c411332-26fb-4f95-921d-55fa469da370 service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:15.469302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c411332-26fb-4f95-921d-55fa469da370 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:15.481497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c411332-26fb-4f95-921d-55fa469da370 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:15.483952 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c411332-26fb-4f95-921d-55fa469da370 service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:15.515333 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1395/2785] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 08:03:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 08:03:17.655523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e63912-f682-4b58-9b57-d2e624694df3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:17.657601 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b0e63912-f682-4b58-9b57-d2e624694df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:17.775133 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:17.779953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e63912-f682-4b58-9b57-d2e624694df3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:17.785143 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5fefc3-ae38-469f-9388-3fa1e5faf391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:17.785896 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fefc3-ae38-469f-9388-3fa1e5faf391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:17.786167 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fefc3-ae38-469f-9388-3fa1e5faf391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:17.786432 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fefc3-ae38-469f-9388-3fa1e5faf391 admin admin] PATH_INFO: `/v3/users/8ad6564a2e9a43c2bdb0358622ff3415` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:17.787576 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5fefc3-ae38-469f-9388-3fa1e5faf391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ad6564a2e9a43c2bdb0358622ff3415)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:17.789791 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5fefc3-ae38-469f-9388-3fa1e5faf391 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:17.871044 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4a5fefc3-ae38-469f-9388-3fa1e5faf391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ad6564a2e9a43c2bdb0358622ff3415'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:17.899709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4a5fefc3-ae38-469f-9388-3fa1e5faf391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ad6564a2e9a43c2bdb0358622ff3415'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:17.925081 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4a5fefc3-ae38-469f-9388-3fa1e5faf391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ad6564a2e9a43c2bdb0358622ff3415'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:17.925773 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4a5fefc3-ae38-469f-9388-3fa1e5faf391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ad6564a2e9a43c2bdb0358622ff3415'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:17.935463 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4a5fefc3-ae38-469f-9388-3fa1e5faf391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ad6564a2e9a43c2bdb0358622ff3415'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:17.942695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1391/2786] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:03:17 2022] DELETE /identity/v3/users/8ad6564a2e9a43c2bdb0358622ff3415 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:18.184537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c411332-26fb-4f95-921d-55fa469da370 service swift] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:18.201640 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c411332-26fb-4f95-921d-55fa469da370 service swift] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:18.229527 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:18.329688 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc8aba-b1c4-44de-ba24-129265bc9a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:18.330247 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc8aba-b1c4-44de-ba24-129265bc9a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:18.330890 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc8aba-b1c4-44de-ba24-129265bc9a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:18.331160 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc8aba-b1c4-44de-ba24-129265bc9a02 admin admin] PATH_INFO: `/v3/projects/53d04fe2eaaf43fc82d82f344faea612` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:18.332971 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc8aba-b1c4-44de-ba24-129265bc9a02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53d04fe2eaaf43fc82d82f344faea612)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:18.336688 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc8aba-b1c4-44de-ba24-129265bc9a02 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:18.430997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1fbc8aba-b1c4-44de-ba24-129265bc9a02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53d04fe2eaaf43fc82d82f344faea612', 'request_id': 'req-1fbc8aba-b1c4-44de-ba24-129265bc9a02'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:18.442224 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1fbc8aba-b1c4-44de-ba24-129265bc9a02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53d04fe2eaaf43fc82d82f344faea612', 'request_id': 'req-1fbc8aba-b1c4-44de-ba24-129265bc9a02'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:18.459401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1fbc8aba-b1c4-44de-ba24-129265bc9a02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53d04fe2eaaf43fc82d82f344faea612', 'request_id': 'req-1fbc8aba-b1c4-44de-ba24-129265bc9a02'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:18.477874 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1fbc8aba-b1c4-44de-ba24-129265bc9a02 admin admin] The token cache is being invalidate because project 53d04fe2eaaf43fc82d82f344faea612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:18.478344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1fbc8aba-b1c4-44de-ba24-129265bc9a02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:18.479127 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1fbc8aba-b1c4-44de-ba24-129265bc9a02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:18.481754 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1396/2787] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:18 2022] DELETE /identity/v3/projects/53d04fe2eaaf43fc82d82f344faea612 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:03:18.486334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5fefc3-ae38-469f-9388-3fa1e5faf391 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:18.487867 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4a5fefc3-ae38-469f-9388-3fa1e5faf391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:18.502860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5fefc3-ae38-469f-9388-3fa1e5faf391 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:18.589023 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90704e1-e600-4d6f-bdd1-808eeedafde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:18.589712 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90704e1-e600-4d6f-bdd1-808eeedafde4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:18.590680 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90704e1-e600-4d6f-bdd1-808eeedafde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:18.591187 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90704e1-e600-4d6f-bdd1-808eeedafde4 admin admin] PATH_INFO: `/v3/users/db13c6e71b31467da98c6ad3da63e063` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:18.592547 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90704e1-e600-4d6f-bdd1-808eeedafde4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db13c6e71b31467da98c6ad3da63e063)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:18.594543 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90704e1-e600-4d6f-bdd1-808eeedafde4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:18.658126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e90704e1-e600-4d6f-bdd1-808eeedafde4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db13c6e71b31467da98c6ad3da63e063'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:18.672472 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e90704e1-e600-4d6f-bdd1-808eeedafde4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db13c6e71b31467da98c6ad3da63e063'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:18.682189 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e90704e1-e600-4d6f-bdd1-808eeedafde4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db13c6e71b31467da98c6ad3da63e063'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:18.683503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e90704e1-e600-4d6f-bdd1-808eeedafde4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db13c6e71b31467da98c6ad3da63e063'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:18.697535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:18.700915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e90704e1-e600-4d6f-bdd1-808eeedafde4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db13c6e71b31467da98c6ad3da63e063'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:18.709472 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1392/2788] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:03:18 2022] DELETE /identity/v3/users/db13c6e71b31467da98c6ad3da63e063 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:18.897267 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc8aba-b1c4-44de-ba24-129265bc9a02 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:18.898836 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1fbc8aba-b1c4-44de-ba24-129265bc9a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:18.915136 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc8aba-b1c4-44de-ba24-129265bc9a02 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:18.989360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:18.990560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:18.991076 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:18.991417 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e admin admin] PATH_INFO: `/v3/projects/62c1491c506848a188fe63ad13c7cf47` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:18.992746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=62c1491c506848a188fe63ad13c7cf47)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:18.994929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:19.044162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:19.047334 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62c1491c506848a188fe63ad13c7cf47', 'request_id': 'req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:19.054846 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62c1491c506848a188fe63ad13c7cf47', 'request_id': 'req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:19.065066 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62c1491c506848a188fe63ad13c7cf47', 'request_id': 'req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:19.078803 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e admin admin] The token cache is being invalidate because project 62c1491c506848a188fe63ad13c7cf47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:19.079102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:19.079719 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:19.081405 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1397/2789] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:18 2022] DELETE /identity/v3/projects/62c1491c506848a188fe63ad13c7cf47 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:03:26.805346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90704e1-e600-4d6f-bdd1-808eeedafde4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:26.806920 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e90704e1-e600-4d6f-bdd1-808eeedafde4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:26.821799 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90704e1-e600-4d6f-bdd1-808eeedafde4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:26.897452 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570502f-d4a4-46a9-bd10-b08e94113638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:26.897983 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570502f-d4a4-46a9-bd10-b08e94113638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:26.898262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570502f-d4a4-46a9-bd10-b08e94113638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:26.898535 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570502f-d4a4-46a9-bd10-b08e94113638 admin admin] PATH_INFO: `/v3/users/507fccc508664f81a6e4fd639302343d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:26.899653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570502f-d4a4-46a9-bd10-b08e94113638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=507fccc508664f81a6e4fd639302343d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:26.901954 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570502f-d4a4-46a9-bd10-b08e94113638 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:26.959241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e570502f-d4a4-46a9-bd10-b08e94113638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '507fccc508664f81a6e4fd639302343d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:26.973019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e570502f-d4a4-46a9-bd10-b08e94113638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '507fccc508664f81a6e4fd639302343d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:26.981642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e570502f-d4a4-46a9-bd10-b08e94113638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '507fccc508664f81a6e4fd639302343d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:26.982309 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e570502f-d4a4-46a9-bd10-b08e94113638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '507fccc508664f81a6e4fd639302343d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:26.990012 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e570502f-d4a4-46a9-bd10-b08e94113638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '507fccc508664f81a6e4fd639302343d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:26.997645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1393/2790] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:03:26 2022] DELETE /identity/v3/users/507fccc508664f81a6e4fd639302343d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:27.209641 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:27.211091 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:27.229862 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd891d9-7c3c-4052-a9c1-5be48eb4fe6e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:27.292766 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d231b-48b7-4188-b75b-395a504c7ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:27.294203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d231b-48b7-4188-b75b-395a504c7ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:27.294473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d231b-48b7-4188-b75b-395a504c7ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:27.294740 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d231b-48b7-4188-b75b-395a504c7ac7 admin admin] PATH_INFO: `/v3/projects/f2376e070e2a4ee19033a277c535c95f` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:27.295933 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d231b-48b7-4188-b75b-395a504c7ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2376e070e2a4ee19033a277c535c95f)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:27.298653 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d231b-48b7-4188-b75b-395a504c7ac7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:27.355223 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-192d231b-48b7-4188-b75b-395a504c7ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2376e070e2a4ee19033a277c535c95f', 'request_id': 'req-192d231b-48b7-4188-b75b-395a504c7ac7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:27.361873 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-192d231b-48b7-4188-b75b-395a504c7ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2376e070e2a4ee19033a277c535c95f', 'request_id': 'req-192d231b-48b7-4188-b75b-395a504c7ac7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:27.368972 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-192d231b-48b7-4188-b75b-395a504c7ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2376e070e2a4ee19033a277c535c95f', 'request_id': 'req-192d231b-48b7-4188-b75b-395a504c7ac7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:27.382269 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-192d231b-48b7-4188-b75b-395a504c7ac7 admin admin] The token cache is being invalidate because project f2376e070e2a4ee19033a277c535c95f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:27.382645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-192d231b-48b7-4188-b75b-395a504c7ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:27.383383 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-192d231b-48b7-4188-b75b-395a504c7ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:27.385676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1398/2791] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:27 2022] DELETE /identity/v3/projects/f2376e070e2a4ee19033a277c535c95f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:27.389220 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570502f-d4a4-46a9-bd10-b08e94113638 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:27.390853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e570502f-d4a4-46a9-bd10-b08e94113638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:27.406406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570502f-d4a4-46a9-bd10-b08e94113638 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:27.469676 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f81298-5c79-4e7f-a2a1-5e8ab51d6255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:27.470320 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f81298-5c79-4e7f-a2a1-5e8ab51d6255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:27.470802 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f81298-5c79-4e7f-a2a1-5e8ab51d6255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:27.470992 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f81298-5c79-4e7f-a2a1-5e8ab51d6255 admin admin] PATH_INFO: `/v3/users/1f5b947a8aa34d6992d9cafa4af25137` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:27.472364 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f81298-5c79-4e7f-a2a1-5e8ab51d6255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f5b947a8aa34d6992d9cafa4af25137)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:27.474486 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f81298-5c79-4e7f-a2a1-5e8ab51d6255 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:27.498720 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:27.564688 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-43f81298-5c79-4e7f-a2a1-5e8ab51d6255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f5b947a8aa34d6992d9cafa4af25137'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:27.578948 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-43f81298-5c79-4e7f-a2a1-5e8ab51d6255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f5b947a8aa34d6992d9cafa4af25137'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:27.587029 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-43f81298-5c79-4e7f-a2a1-5e8ab51d6255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f5b947a8aa34d6992d9cafa4af25137'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:27.587722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-43f81298-5c79-4e7f-a2a1-5e8ab51d6255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f5b947a8aa34d6992d9cafa4af25137'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:27.597345 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-43f81298-5c79-4e7f-a2a1-5e8ab51d6255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f5b947a8aa34d6992d9cafa4af25137'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:27.603826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1394/2792] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:03:27 2022] DELETE /identity/v3/users/1f5b947a8aa34d6992d9cafa4af25137 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:03:27.809619 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d231b-48b7-4188-b75b-395a504c7ac7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:27.812494 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-192d231b-48b7-4188-b75b-395a504c7ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:27.836994 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d231b-48b7-4188-b75b-395a504c7ac7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:27.898852 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725fb246-3b5c-4fa7-9e5c-a3ecc41841d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:27.899506 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fb246-3b5c-4fa7-9e5c-a3ecc41841d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:27.899808 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fb246-3b5c-4fa7-9e5c-a3ecc41841d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:27.900097 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fb246-3b5c-4fa7-9e5c-a3ecc41841d1 admin admin] PATH_INFO: `/v3/projects/4bd4cc8a886b4ed1a448cc9b08953d38` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:27.901333 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725fb246-3b5c-4fa7-9e5c-a3ecc41841d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd4cc8a886b4ed1a448cc9b08953d38)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:27.904292 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725fb246-3b5c-4fa7-9e5c-a3ecc41841d1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:27.911484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:27.966101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-725fb246-3b5c-4fa7-9e5c-a3ecc41841d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd4cc8a886b4ed1a448cc9b08953d38', 'request_id': 'req-725fb246-3b5c-4fa7-9e5c-a3ecc41841d1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:27.972517 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-725fb246-3b5c-4fa7-9e5c-a3ecc41841d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd4cc8a886b4ed1a448cc9b08953d38', 'request_id': 'req-725fb246-3b5c-4fa7-9e5c-a3ecc41841d1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:27.979122 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-725fb246-3b5c-4fa7-9e5c-a3ecc41841d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd4cc8a886b4ed1a448cc9b08953d38', 'request_id': 'req-725fb246-3b5c-4fa7-9e5c-a3ecc41841d1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:27.992182 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-725fb246-3b5c-4fa7-9e5c-a3ecc41841d1 admin admin] The token cache is being invalidate because project 4bd4cc8a886b4ed1a448cc9b08953d38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:27.992467 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-725fb246-3b5c-4fa7-9e5c-a3ecc41841d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:27.993144 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-725fb246-3b5c-4fa7-9e5c-a3ecc41841d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:27.994649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1399/2793] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:27 2022] DELETE /identity/v3/projects/4bd4cc8a886b4ed1a448cc9b08953d38 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:27.999752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e8c6d-afb4-4032-80ca-442ad8702094 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:28.000661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e8c6d-afb4-4032-80ca-442ad8702094 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:28.001234 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e8c6d-afb4-4032-80ca-442ad8702094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:28.070996 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2e2e8c6d-afb4-4032-80ca-442ad8702094 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:28.072419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2e2e8c6d-afb4-4032-80ca-442ad8702094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:28.151868 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1395/2794] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:03:28.218594 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54398df-35e7-4363-a055-786b254dab59 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:28.219084 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54398df-35e7-4363-a055-786b254dab59 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:28.219187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54398df-35e7-4363-a055-786b254dab59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:28.242537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d54398df-35e7-4363-a055-786b254dab59 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:28.243757 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d54398df-35e7-4363-a055-786b254dab59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:28.254706 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1400/2795] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:28.258542 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e8c6d-afb4-4032-80ca-442ad8702094 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:28.268993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e8c6d-afb4-4032-80ca-442ad8702094 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:28.274600 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3296e-3d6f-4727-bd19-310ff69d9294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:28.275218 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3296e-3d6f-4727-bd19-310ff69d9294 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:28.275502 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3296e-3d6f-4727-bd19-310ff69d9294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:28.275776 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3296e-3d6f-4727-bd19-310ff69d9294 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:28.276447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca3296e-3d6f-4727-bd19-310ff69d9294 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:03:28.276749 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca3296e-3d6f-4727-bd19-310ff69d9294 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:28.279025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca3296e-3d6f-4727-bd19-310ff69d9294 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:28.292112 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1396/2796] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:03:28.297106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54398df-35e7-4363-a055-786b254dab59 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:28.307020 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54398df-35e7-4363-a055-786b254dab59 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:28.312484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d0ba8-2680-4b73-9c08-84bdbeb75b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:28.313077 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d0ba8-2680-4b73-9c08-84bdbeb75b52 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:28.313385 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d0ba8-2680-4b73-9c08-84bdbeb75b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:28.313669 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d0ba8-2680-4b73-9c08-84bdbeb75b52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:28.314319 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d0ba8-2680-4b73-9c08-84bdbeb75b52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:28.316302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d0ba8-2680-4b73-9c08-84bdbeb75b52 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:28.357535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1401/2797] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:03:28 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:03:28.361716 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3296e-3d6f-4727-bd19-310ff69d9294 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:28.374620 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3296e-3d6f-4727-bd19-310ff69d9294 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:28.432377 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:28.446463 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b303fb9-656c-45f7-83d5-7bedb6e58d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:28.447351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b303fb9-656c-45f7-83d5-7bedb6e58d45 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:28.448002 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b303fb9-656c-45f7-83d5-7bedb6e58d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:28.448387 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b303fb9-656c-45f7-83d5-7bedb6e58d45 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:28.449809 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b303fb9-656c-45f7-83d5-7bedb6e58d45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:28.452645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b303fb9-656c-45f7-83d5-7bedb6e58d45 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:28.479278 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1397/2798] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:03:28 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:03:28.484037 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d0ba8-2680-4b73-9c08-84bdbeb75b52 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:28.495680 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d0ba8-2680-4b73-9c08-84bdbeb75b52 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:28.501000 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f095e6-f385-4ba1-9653-4a8c75fe4234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:28.501621 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f095e6-f385-4ba1-9653-4a8c75fe4234 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:28.501937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f095e6-f385-4ba1-9653-4a8c75fe4234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:28.502161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f095e6-f385-4ba1-9653-4a8c75fe4234 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:28.502820 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f095e6-f385-4ba1-9653-4a8c75fe4234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:28.504468 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f095e6-f385-4ba1-9653-4a8c75fe4234 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:28.517028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1402/2799] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:28.521045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b303fb9-656c-45f7-83d5-7bedb6e58d45 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:28.542117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b303fb9-656c-45f7-83d5-7bedb6e58d45 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:28.550305 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e71f4-ca7b-402a-9b91-c2977eb92a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:28.551126 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e71f4-ca7b-402a-9b91-c2977eb92a2c admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:28.551418 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e71f4-ca7b-402a-9b91-c2977eb92a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:28.551952 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e71f4-ca7b-402a-9b91-c2977eb92a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:28.552647 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e71f4-ca7b-402a-9b91-c2977eb92a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:28.554404 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e71f4-ca7b-402a-9b91-c2977eb92a2c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:28.568536 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1398/2800] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:03:28.572421 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f095e6-f385-4ba1-9653-4a8c75fe4234 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:28.582796 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f095e6-f385-4ba1-9653-4a8c75fe4234 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:28.587712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a0be4-7f53-461a-aab6-1ae7f6a2dca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:28.588391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a0be4-7f53-461a-aab6-1ae7f6a2dca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:28.588667 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a0be4-7f53-461a-aab6-1ae7f6a2dca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:28.588940 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a0be4-7f53-461a-aab6-1ae7f6a2dca1 admin admin] PATH_INFO: `/v3/projects/e5225e76b6f74a7b865df8d091bd41e7/users/fd4fbb91d1dc4e1f842203352a008cc5/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:28.609125 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a0be4-7f53-461a-aab6-1ae7f6a2dca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5225e76b6f74a7b865df8d091bd41e7, user_id=fd4fbb91d1dc4e1f842203352a008cc5, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:28.611875 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a0be4-7f53-461a-aab6-1ae7f6a2dca1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:28.627870 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1403/2801] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:03:28 2022] PUT /identity/v3/projects/e5225e76b6f74a7b865df8d091bd41e7/users/fd4fbb91d1dc4e1f842203352a008cc5/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:28.638113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2720e1c-0622-471e-ba7e-8a4f7fda1549 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:28.638402 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2720e1c-0622-471e-ba7e-8a4f7fda1549 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:28.638687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2720e1c-0622-471e-ba7e-8a4f7fda1549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:28.667175 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c2720e1c-0622-471e-ba7e-8a4f7fda1549 None None] MFA Rules not processed for user `fd4fbb91d1dc4e1f842203352a008cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:28.669141 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c2720e1c-0622-471e-ba7e-8a4f7fda1549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:28.742257 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1399/2802] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:03:28.752890 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a0be4-7f53-461a-aab6-1ae7f6a2dca1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:28.755099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7d6a0be4-7f53-461a-aab6-1ae7f6a2dca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:28.817421 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a0be4-7f53-461a-aab6-1ae7f6a2dca1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:28.821487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d05ec-c7e2-43a2-9cba-7d7eb69f4cd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:28.822090 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d05ec-c7e2-43a2-9cba-7d7eb69f4cd4 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:28.822400 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d05ec-c7e2-43a2-9cba-7d7eb69f4cd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:28.822641 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d05ec-c7e2-43a2-9cba-7d7eb69f4cd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:28.832856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d05ec-c7e2-43a2-9cba-7d7eb69f4cd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:28.834528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d05ec-c7e2-43a2-9cba-7d7eb69f4cd4 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:28.844839 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1404/2803] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:37.485859 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2720e1c-0622-471e-ba7e-8a4f7fda1549 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:37.564348 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2720e1c-0622-471e-ba7e-8a4f7fda1549 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:37.569909 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8e701-7eae-4b4e-9865-2d7e95475faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:37.570570 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8e701-7eae-4b4e-9865-2d7e95475faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:37.570851 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8e701-7eae-4b4e-9865-2d7e95475faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:37.571128 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8e701-7eae-4b4e-9865-2d7e95475faa admin admin] PATH_INFO: `/v3/users/fd4fbb91d1dc4e1f842203352a008cc5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:37.572323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8e701-7eae-4b4e-9865-2d7e95475faa admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd4fbb91d1dc4e1f842203352a008cc5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:37.574829 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8e701-7eae-4b4e-9865-2d7e95475faa admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:37.628299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1df8e701-7eae-4b4e-9865-2d7e95475faa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd4fbb91d1dc4e1f842203352a008cc5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:37.643158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:37.647075 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1df8e701-7eae-4b4e-9865-2d7e95475faa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd4fbb91d1dc4e1f842203352a008cc5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:37.660672 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1df8e701-7eae-4b4e-9865-2d7e95475faa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd4fbb91d1dc4e1f842203352a008cc5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:37.662670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1df8e701-7eae-4b4e-9865-2d7e95475faa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd4fbb91d1dc4e1f842203352a008cc5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:37.672181 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1df8e701-7eae-4b4e-9865-2d7e95475faa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd4fbb91d1dc4e1f842203352a008cc5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:37.679722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1400/2804] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:03:37 2022] DELETE /identity/v3/users/fd4fbb91d1dc4e1f842203352a008cc5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:37.684682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d2a52-c8d5-42e9-81b8-aafb6d1555cf None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:37.685187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d2a52-c8d5-42e9-81b8-aafb6d1555cf None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:37.685502 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d2a52-c8d5-42e9-81b8-aafb6d1555cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:37.713271 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-449d2a52-c8d5-42e9-81b8-aafb6d1555cf None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:37.717317 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-449d2a52-c8d5-42e9-81b8-aafb6d1555cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:37.751199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:37.795071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1405/2805] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:37.805766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8e701-7eae-4b4e-9865-2d7e95475faa admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:37.807461 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1df8e701-7eae-4b4e-9865-2d7e95475faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:37.909751 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8e701-7eae-4b4e-9865-2d7e95475faa admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:37.914093 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75069218-b0d7-4794-8bf6-2bc9e54953e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:37.914771 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75069218-b0d7-4794-8bf6-2bc9e54953e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:37.915077 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75069218-b0d7-4794-8bf6-2bc9e54953e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:37.915377 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75069218-b0d7-4794-8bf6-2bc9e54953e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:37.926323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75069218-b0d7-4794-8bf6-2bc9e54953e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:37.927786 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75069218-b0d7-4794-8bf6-2bc9e54953e0 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:37.939938 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1401/2806] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:38.303989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d2a52-c8d5-42e9-81b8-aafb6d1555cf None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:38.305601 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-449d2a52-c8d5-42e9-81b8-aafb6d1555cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:38.319858 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d2a52-c8d5-42e9-81b8-aafb6d1555cf None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:38.337052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37283c8-1a20-4bbd-86db-5d580278025d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:38.337722 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37283c8-1a20-4bbd-86db-5d580278025d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:38.338200 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37283c8-1a20-4bbd-86db-5d580278025d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:38.338467 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37283c8-1a20-4bbd-86db-5d580278025d admin admin] PATH_INFO: `/v3/projects/e5225e76b6f74a7b865df8d091bd41e7` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:38.340305 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37283c8-1a20-4bbd-86db-5d580278025d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5225e76b6f74a7b865df8d091bd41e7)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:38.342078 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37283c8-1a20-4bbd-86db-5d580278025d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:38.408391 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b37283c8-1a20-4bbd-86db-5d580278025d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5225e76b6f74a7b865df8d091bd41e7', 'request_id': 'req-b37283c8-1a20-4bbd-86db-5d580278025d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:38.421705 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b37283c8-1a20-4bbd-86db-5d580278025d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5225e76b6f74a7b865df8d091bd41e7', 'request_id': 'req-b37283c8-1a20-4bbd-86db-5d580278025d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:38.429435 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b37283c8-1a20-4bbd-86db-5d580278025d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5225e76b6f74a7b865df8d091bd41e7', 'request_id': 'req-b37283c8-1a20-4bbd-86db-5d580278025d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:38.444505 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b37283c8-1a20-4bbd-86db-5d580278025d admin admin] The token cache is being invalidate because project e5225e76b6f74a7b865df8d091bd41e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:38.444925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b37283c8-1a20-4bbd-86db-5d580278025d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:38.445717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b37283c8-1a20-4bbd-86db-5d580278025d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:38.450841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1406/2807] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:38 2022] DELETE /identity/v3/projects/e5225e76b6f74a7b865df8d091bd41e7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:38.452883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce99cc6-0d3c-4871-9e57-5afc15725a7f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:38.453206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce99cc6-0d3c-4871-9e57-5afc15725a7f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:38.453474 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce99cc6-0d3c-4871-9e57-5afc15725a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:38.477669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1ce99cc6-0d3c-4871-9e57-5afc15725a7f None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:38.479161 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1ce99cc6-0d3c-4871-9e57-5afc15725a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:38.545117 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1402/2808] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:38.553868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dc882-a384-4ffe-b547-0d6b1b7d6a96 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:38.554167 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dc882-a384-4ffe-b547-0d6b1b7d6a96 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:38.554513 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dc882-a384-4ffe-b547-0d6b1b7d6a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:38.580866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-785dc882-a384-4ffe-b547-0d6b1b7d6a96 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:38.582082 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-785dc882-a384-4ffe-b547-0d6b1b7d6a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:38.590844 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1407/2809] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:38.595004 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce99cc6-0d3c-4871-9e57-5afc15725a7f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:38.604924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce99cc6-0d3c-4871-9e57-5afc15725a7f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:38.610498 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6d9d0-f655-4eaf-931b-7c6efe6a8af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:38.611106 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6d9d0-f655-4eaf-931b-7c6efe6a8af8 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:38.611383 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6d9d0-f655-4eaf-931b-7c6efe6a8af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:38.611662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6d9d0-f655-4eaf-931b-7c6efe6a8af8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:38.612331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6d9d0-f655-4eaf-931b-7c6efe6a8af8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:03:38.612632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6d9d0-f655-4eaf-931b-7c6efe6a8af8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:38.614209 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6d9d0-f655-4eaf-931b-7c6efe6a8af8 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:38.626158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1403/2810] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:03:38.630489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dc882-a384-4ffe-b547-0d6b1b7d6a96 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:38.640480 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dc882-a384-4ffe-b547-0d6b1b7d6a96 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:38.645319 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e22f3-fa24-49b6-ad7f-7bf3e16e005e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:38.645924 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e22f3-fa24-49b6-ad7f-7bf3e16e005e admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:38.646204 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e22f3-fa24-49b6-ad7f-7bf3e16e005e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:38.646477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e22f3-fa24-49b6-ad7f-7bf3e16e005e admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:38.647102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e22f3-fa24-49b6-ad7f-7bf3e16e005e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:38.649075 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e22f3-fa24-49b6-ad7f-7bf3e16e005e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:38.679171 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1408/2811] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:03:38 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:03:38.688405 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6d9d0-f655-4eaf-931b-7c6efe6a8af8 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:38.700533 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6d9d0-f655-4eaf-931b-7c6efe6a8af8 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:38.749556 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:38.770615 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4cdf1e-73a3-41c4-a0ce-192a7d970b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:38.771274 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cdf1e-73a3-41c4-a0ce-192a7d970b33 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:38.771567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cdf1e-73a3-41c4-a0ce-192a7d970b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:38.771819 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cdf1e-73a3-41c4-a0ce-192a7d970b33 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:38.772495 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4cdf1e-73a3-41c4-a0ce-192a7d970b33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:38.774787 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4cdf1e-73a3-41c4-a0ce-192a7d970b33 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:38.804211 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1404/2812] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:03:38 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:03:38.808202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e22f3-fa24-49b6-ad7f-7bf3e16e005e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:38.819780 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e22f3-fa24-49b6-ad7f-7bf3e16e005e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:38.824706 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d033a90-02c5-4fa7-b378-582974ba2389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:38.825336 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d033a90-02c5-4fa7-b378-582974ba2389 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:38.825616 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d033a90-02c5-4fa7-b378-582974ba2389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:38.825891 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d033a90-02c5-4fa7-b378-582974ba2389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:38.826536 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d033a90-02c5-4fa7-b378-582974ba2389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:38.828376 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d033a90-02c5-4fa7-b378-582974ba2389 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:38.841201 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1409/2813] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:38.845146 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4cdf1e-73a3-41c4-a0ce-192a7d970b33 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:38.860016 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4cdf1e-73a3-41c4-a0ce-192a7d970b33 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:38.868474 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56450f-9e05-4622-b3c5-bd270b95a90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:38.869277 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56450f-9e05-4622-b3c5-bd270b95a90f admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:38.869554 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56450f-9e05-4622-b3c5-bd270b95a90f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:38.869822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56450f-9e05-4622-b3c5-bd270b95a90f admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:38.870468 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56450f-9e05-4622-b3c5-bd270b95a90f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:38.872650 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56450f-9e05-4622-b3c5-bd270b95a90f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:38.884273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1405/2814] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:38.889523 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d033a90-02c5-4fa7-b378-582974ba2389 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:38.901883 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d033a90-02c5-4fa7-b378-582974ba2389 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:38.907144 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c7a2c-a9da-44dd-a5fd-d2f9c6548b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:38.907871 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c7a2c-a9da-44dd-a5fd-d2f9c6548b59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:38.908183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c7a2c-a9da-44dd-a5fd-d2f9c6548b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:38.908452 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c7a2c-a9da-44dd-a5fd-d2f9c6548b59 admin admin] PATH_INFO: `/v3/projects/e7ce484515e641eba374170d018ecf49/users/d75e1e36aac84ac09d549fc5fd165e06/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:38.930036 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c7a2c-a9da-44dd-a5fd-d2f9c6548b59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ce484515e641eba374170d018ecf49, user_id=d75e1e36aac84ac09d549fc5fd165e06, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:38.932781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c7a2c-a9da-44dd-a5fd-d2f9c6548b59 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:38.950700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1410/2815] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:03:38 2022] PUT /identity/v3/projects/e7ce484515e641eba374170d018ecf49/users/d75e1e36aac84ac09d549fc5fd165e06/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:03:38.964000 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39631f2f-eb03-4ec6-b630-40c99b25e396 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:38.964407 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39631f2f-eb03-4ec6-b630-40c99b25e396 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:38.964744 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39631f2f-eb03-4ec6-b630-40c99b25e396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:38.995634 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-39631f2f-eb03-4ec6-b630-40c99b25e396 None None] MFA Rules not processed for user `d75e1e36aac84ac09d549fc5fd165e06`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:38.997429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-39631f2f-eb03-4ec6-b630-40c99b25e396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:39.091581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1406/2816] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:39.102571 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c7a2c-a9da-44dd-a5fd-d2f9c6548b59 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:39.106027 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5e4c7a2c-a9da-44dd-a5fd-d2f9c6548b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:39.181596 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c7a2c-a9da-44dd-a5fd-d2f9c6548b59 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:39.188308 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ddb60-f728-4f74-b31e-0d485966bfdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:39.189199 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ddb60-f728-4f74-b31e-0d485966bfdf service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:39.189645 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ddb60-f728-4f74-b31e-0d485966bfdf service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:39.190089 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ddb60-f728-4f74-b31e-0d485966bfdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:39.206836 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:39.212700 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ddb60-f728-4f74-b31e-0d485966bfdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:39.214823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ddb60-f728-4f74-b31e-0d485966bfdf service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:39.230574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1411/2817] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:41.503118 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c6a55-fc55-4bd7-ad59-b7cb447dc88d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:41.503608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c6a55-fc55-4bd7-ad59-b7cb447dc88d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:41.503717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c6a55-fc55-4bd7-ad59-b7cb447dc88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:41.532567 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e82c6a55-fc55-4bd7-ad59-b7cb447dc88d None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:41.533773 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e82c6a55-fc55-4bd7-ad59-b7cb447dc88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:41.587377 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1407/2818] 10.222.0.26 () {58 vars in 967 bytes} [Wed Sep 7 08:03:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:41.599880 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ddb60-f728-4f74-b31e-0d485966bfdf service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:41.601266 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b37ddb60-f728-4f74-b31e-0d485966bfdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:41.669218 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ddb60-f728-4f74-b31e-0d485966bfdf service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:41.673983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28074878-898e-43fc-bd42-c8f7fba47650 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:41.674577 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28074878-898e-43fc-bd42-c8f7fba47650 service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:41.674849 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28074878-898e-43fc-bd42-c8f7fba47650 service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:41.675115 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28074878-898e-43fc-bd42-c8f7fba47650 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:41.682894 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28074878-898e-43fc-bd42-c8f7fba47650 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:41.684491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28074878-898e-43fc-bd42-c8f7fba47650 service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:41.692701 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1412/2819] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 08:03:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 08:03:41.941841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c6a55-fc55-4bd7-ad59-b7cb447dc88d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:42.020999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:42.040838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c6a55-fc55-4bd7-ad59-b7cb447dc88d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:42.048639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068007b-96b6-4e59-8472-b9bdea0ac01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:42.049717 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068007b-96b6-4e59-8472-b9bdea0ac01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:42.050001 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068007b-96b6-4e59-8472-b9bdea0ac01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:42.050270 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068007b-96b6-4e59-8472-b9bdea0ac01a admin admin] PATH_INFO: `/v3/users/d75e1e36aac84ac09d549fc5fd165e06` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:42.051436 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068007b-96b6-4e59-8472-b9bdea0ac01a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d75e1e36aac84ac09d549fc5fd165e06)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:42.053475 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068007b-96b6-4e59-8472-b9bdea0ac01a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:42.106937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9068007b-96b6-4e59-8472-b9bdea0ac01a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd75e1e36aac84ac09d549fc5fd165e06'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:42.119230 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9068007b-96b6-4e59-8472-b9bdea0ac01a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd75e1e36aac84ac09d549fc5fd165e06'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:42.126759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9068007b-96b6-4e59-8472-b9bdea0ac01a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd75e1e36aac84ac09d549fc5fd165e06'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:42.127373 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9068007b-96b6-4e59-8472-b9bdea0ac01a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd75e1e36aac84ac09d549fc5fd165e06'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:42.134348 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9068007b-96b6-4e59-8472-b9bdea0ac01a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd75e1e36aac84ac09d549fc5fd165e06'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:42.143239 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1408/2820] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:03:41 2022] DELETE /identity/v3/users/d75e1e36aac84ac09d549fc5fd165e06 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:42.147370 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff692c-4e8a-46f4-8449-c6ace3e7907f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:42.147667 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff692c-4e8a-46f4-8449-c6ace3e7907f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:42.148012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff692c-4e8a-46f4-8449-c6ace3e7907f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:42.176461 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-26ff692c-4e8a-46f4-8449-c6ace3e7907f None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:42.177666 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-26ff692c-4e8a-46f4-8449-c6ace3e7907f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:42.237285 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1413/2821] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:03:42.247667 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068007b-96b6-4e59-8472-b9bdea0ac01a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:42.249080 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9068007b-96b6-4e59-8472-b9bdea0ac01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:42.306089 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068007b-96b6-4e59-8472-b9bdea0ac01a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:42.310550 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68a29e-23db-4f39-b10e-6382b43af964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:42.311434 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68a29e-23db-4f39-b10e-6382b43af964 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:42.311707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68a29e-23db-4f39-b10e-6382b43af964 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:42.311974 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68a29e-23db-4f39-b10e-6382b43af964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:42.321318 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68a29e-23db-4f39-b10e-6382b43af964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:42.322713 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68a29e-23db-4f39-b10e-6382b43af964 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:42.332838 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1409/2822] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:03:42.600377 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff692c-4e8a-46f4-8449-c6ace3e7907f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:42.601814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-26ff692c-4e8a-46f4-8449-c6ace3e7907f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:42.614497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff692c-4e8a-46f4-8449-c6ace3e7907f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:42.619075 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234c7db-fdfc-4667-9666-fdfcb577f38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:42.619704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234c7db-fdfc-4667-9666-fdfcb577f38e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:42.619978 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234c7db-fdfc-4667-9666-fdfcb577f38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:42.620274 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234c7db-fdfc-4667-9666-fdfcb577f38e admin admin] PATH_INFO: `/v3/projects/e7ce484515e641eba374170d018ecf49` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:42.621530 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234c7db-fdfc-4667-9666-fdfcb577f38e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7ce484515e641eba374170d018ecf49)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:42.623444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234c7db-fdfc-4667-9666-fdfcb577f38e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:42.670491 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1234c7db-fdfc-4667-9666-fdfcb577f38e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7ce484515e641eba374170d018ecf49', 'request_id': 'req-1234c7db-fdfc-4667-9666-fdfcb577f38e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:42.675890 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1234c7db-fdfc-4667-9666-fdfcb577f38e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7ce484515e641eba374170d018ecf49', 'request_id': 'req-1234c7db-fdfc-4667-9666-fdfcb577f38e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:42.682005 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1234c7db-fdfc-4667-9666-fdfcb577f38e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7ce484515e641eba374170d018ecf49', 'request_id': 'req-1234c7db-fdfc-4667-9666-fdfcb577f38e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:42.693901 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1234c7db-fdfc-4667-9666-fdfcb577f38e admin admin] The token cache is being invalidate because project e7ce484515e641eba374170d018ecf49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:42.694187 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1234c7db-fdfc-4667-9666-fdfcb577f38e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:42.694800 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1234c7db-fdfc-4667-9666-fdfcb577f38e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:42.696284 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1414/2823] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:42 2022] DELETE /identity/v3/projects/e7ce484515e641eba374170d018ecf49 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:42.702895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf763b4-fc9c-4595-8296-a17fbbe9302a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:42.703365 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf763b4-fc9c-4595-8296-a17fbbe9302a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:42.703478 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf763b4-fc9c-4595-8296-a17fbbe9302a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:42.723796 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1cf763b4-fc9c-4595-8296-a17fbbe9302a None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:42.725010 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1cf763b4-fc9c-4595-8296-a17fbbe9302a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:42.782895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1410/2824] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:42.790471 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbc707-2c04-47b1-8fda-7b14f573a6f9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:42.790874 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbc707-2c04-47b1-8fda-7b14f573a6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:42.791158 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbc707-2c04-47b1-8fda-7b14f573a6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:42.810746 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b8fbc707-2c04-47b1-8fda-7b14f573a6f9 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:42.812240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b8fbc707-2c04-47b1-8fda-7b14f573a6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:42.820007 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1415/2825] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:42.824721 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf763b4-fc9c-4595-8296-a17fbbe9302a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:42.834214 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf763b4-fc9c-4595-8296-a17fbbe9302a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:42.838955 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2d391-8b72-4639-b4ed-99fcf7b248c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:42.839587 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2d391-8b72-4639-b4ed-99fcf7b248c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:42.840655 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2d391-8b72-4639-b4ed-99fcf7b248c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:42.841121 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2d391-8b72-4639-b4ed-99fcf7b248c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:42.841787 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2d391-8b72-4639-b4ed-99fcf7b248c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:03:42.842090 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2d391-8b72-4639-b4ed-99fcf7b248c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:42.844304 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2d391-8b72-4639-b4ed-99fcf7b248c2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:42.855636 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1411/2826] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:03:42.859989 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbc707-2c04-47b1-8fda-7b14f573a6f9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:42.864551 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:42.871003 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbc707-2c04-47b1-8fda-7b14f573a6f9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:42.876694 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91bdc7a-d365-48bc-bb5d-2b6eebe1c452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:42.877282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bdc7a-d365-48bc-bb5d-2b6eebe1c452 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:42.877553 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bdc7a-d365-48bc-bb5d-2b6eebe1c452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:42.877816 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bdc7a-d365-48bc-bb5d-2b6eebe1c452 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:42.878430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91bdc7a-d365-48bc-bb5d-2b6eebe1c452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:42.880403 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91bdc7a-d365-48bc-bb5d-2b6eebe1c452 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:42.907150 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1416/2827] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:03:42 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:03:42.911318 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2d391-8b72-4639-b4ed-99fcf7b248c2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:42.916649 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:42.922825 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2d391-8b72-4639-b4ed-99fcf7b248c2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:42.978171 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44306f7c-c63b-43c2-9b08-2e4f968f8ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:42.978704 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44306f7c-c63b-43c2-9b08-2e4f968f8ccd admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:42.978982 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44306f7c-c63b-43c2-9b08-2e4f968f8ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:42.979253 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44306f7c-c63b-43c2-9b08-2e4f968f8ccd admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:42.979883 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44306f7c-c63b-43c2-9b08-2e4f968f8ccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:42.982010 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44306f7c-c63b-43c2-9b08-2e4f968f8ccd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:43.002179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1412/2828] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:03:42 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:03:43.006321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91bdc7a-d365-48bc-bb5d-2b6eebe1c452 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:43.017026 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91bdc7a-d365-48bc-bb5d-2b6eebe1c452 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:43.021487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e0d53-0857-42c3-869d-637e059ff57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:43.022070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e0d53-0857-42c3-869d-637e059ff57a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:43.022344 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e0d53-0857-42c3-869d-637e059ff57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:43.022610 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e0d53-0857-42c3-869d-637e059ff57a admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:43.023261 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e0d53-0857-42c3-869d-637e059ff57a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:43.024871 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e0d53-0857-42c3-869d-637e059ff57a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:43.035520 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1417/2829] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:43.039313 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44306f7c-c63b-43c2-9b08-2e4f968f8ccd admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:43.049618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44306f7c-c63b-43c2-9b08-2e4f968f8ccd admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:43.054148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdccb00-0183-4d10-8362-5616ae7b7972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:43.054722 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdccb00-0183-4d10-8362-5616ae7b7972 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:43.054997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdccb00-0183-4d10-8362-5616ae7b7972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:43.055263 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdccb00-0183-4d10-8362-5616ae7b7972 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:43.055885 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdccb00-0183-4d10-8362-5616ae7b7972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:43.058305 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdccb00-0183-4d10-8362-5616ae7b7972 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:43.070927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1413/2830] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:43.075714 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e0d53-0857-42c3-869d-637e059ff57a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:43.085868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e0d53-0857-42c3-869d-637e059ff57a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:43.091098 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1681f-424a-44be-bdeb-cb5b0f947d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:43.091876 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1681f-424a-44be-bdeb-cb5b0f947d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:43.092029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1681f-424a-44be-bdeb-cb5b0f947d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:43.092330 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1681f-424a-44be-bdeb-cb5b0f947d02 admin admin] PATH_INFO: `/v3/projects/5823101cc155404fbaa66c3b9c191d3d/users/dcb3bd73ea94441e94a8efdb908cc89f/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:43.110198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1681f-424a-44be-bdeb-cb5b0f947d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5823101cc155404fbaa66c3b9c191d3d, user_id=dcb3bd73ea94441e94a8efdb908cc89f, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:43.110198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1681f-424a-44be-bdeb-cb5b0f947d02 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:43.124378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1418/2831] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:03:43 2022] PUT /identity/v3/projects/5823101cc155404fbaa66c3b9c191d3d/users/dcb3bd73ea94441e94a8efdb908cc89f/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:43.135602 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d6793-19ba-4c7f-91b5-82d43e1aa2d0 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:43.135892 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d6793-19ba-4c7f-91b5-82d43e1aa2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:43.136191 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d6793-19ba-4c7f-91b5-82d43e1aa2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:43.165823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-328d6793-19ba-4c7f-91b5-82d43e1aa2d0 None None] MFA Rules not processed for user `dcb3bd73ea94441e94a8efdb908cc89f`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:43.167035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-328d6793-19ba-4c7f-91b5-82d43e1aa2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:43.237532 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1414/2832] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:43.249983 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1681f-424a-44be-bdeb-cb5b0f947d02 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:43.251517 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5bf1681f-424a-44be-bdeb-cb5b0f947d02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:43.340359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1681f-424a-44be-bdeb-cb5b0f947d02 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:43.344721 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f815f-367f-46d7-8cd8-41401e8740a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:43.345347 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f815f-367f-46d7-8cd8-41401e8740a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:43.345619 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f815f-367f-46d7-8cd8-41401e8740a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:43.345886 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f815f-367f-46d7-8cd8-41401e8740a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:43.354859 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f815f-367f-46d7-8cd8-41401e8740a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:43.356487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f815f-367f-46d7-8cd8-41401e8740a5 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:43.365444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1419/2833] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:43.395822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d6793-19ba-4c7f-91b5-82d43e1aa2d0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:43.460020 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d6793-19ba-4c7f-91b5-82d43e1aa2d0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:43.464811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d027152-fbf2-435c-8e90-8b12540ec825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:43.465484 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d027152-fbf2-435c-8e90-8b12540ec825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:43.465811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d027152-fbf2-435c-8e90-8b12540ec825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:43.466084 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d027152-fbf2-435c-8e90-8b12540ec825 admin admin] PATH_INFO: `/v3/users/dcb3bd73ea94441e94a8efdb908cc89f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:43.467241 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d027152-fbf2-435c-8e90-8b12540ec825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcb3bd73ea94441e94a8efdb908cc89f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:43.469202 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d027152-fbf2-435c-8e90-8b12540ec825 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:43.482639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:43.522450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8d027152-fbf2-435c-8e90-8b12540ec825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcb3bd73ea94441e94a8efdb908cc89f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:43.535283 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8d027152-fbf2-435c-8e90-8b12540ec825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcb3bd73ea94441e94a8efdb908cc89f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:43.544441 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8d027152-fbf2-435c-8e90-8b12540ec825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcb3bd73ea94441e94a8efdb908cc89f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:43.545065 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8d027152-fbf2-435c-8e90-8b12540ec825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcb3bd73ea94441e94a8efdb908cc89f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:43.552259 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8d027152-fbf2-435c-8e90-8b12540ec825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcb3bd73ea94441e94a8efdb908cc89f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:43.558937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1415/2834] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:03:43 2022] DELETE /identity/v3/users/dcb3bd73ea94441e94a8efdb908cc89f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:43.563769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6dba7-2168-4779-b385-dc3279d1adf7 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:43.564057 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6dba7-2168-4779-b385-dc3279d1adf7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:43.564409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6dba7-2168-4779-b385-dc3279d1adf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:43.586593 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c0d6dba7-2168-4779-b385-dc3279d1adf7 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:43.587824 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c0d6dba7-2168-4779-b385-dc3279d1adf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:43.648392 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d027152-fbf2-435c-8e90-8b12540ec825 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:43.649924 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8d027152-fbf2-435c-8e90-8b12540ec825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:43.651929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1420/2835] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:43.664188 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6dba7-2168-4779-b385-dc3279d1adf7 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:43.665682 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c0d6dba7-2168-4779-b385-dc3279d1adf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:43.667310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d027152-fbf2-435c-8e90-8b12540ec825 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:43.672738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d55a6a-e78f-4a02-9c62-b613fb7d9fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:43.673390 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d55a6a-e78f-4a02-9c62-b613fb7d9fe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:43.673665 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d55a6a-e78f-4a02-9c62-b613fb7d9fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:43.673933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d55a6a-e78f-4a02-9c62-b613fb7d9fe5 admin admin] PATH_INFO: `/v3/users/7a67efdb6af84cfb92df1be6d8345ea9` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:43.675052 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d55a6a-e78f-4a02-9c62-b613fb7d9fe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a67efdb6af84cfb92df1be6d8345ea9)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:43.677003 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d55a6a-e78f-4a02-9c62-b613fb7d9fe5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:43.699936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:43.726616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-55d55a6a-e78f-4a02-9c62-b613fb7d9fe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a67efdb6af84cfb92df1be6d8345ea9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:43.728852 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6dba7-2168-4779-b385-dc3279d1adf7 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:43.732797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4136f-9e2d-4099-940d-98644ce32de9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:43.733450 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4136f-9e2d-4099-940d-98644ce32de9 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:43.733731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4136f-9e2d-4099-940d-98644ce32de9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:43.733999 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4136f-9e2d-4099-940d-98644ce32de9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:43.739078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-55d55a6a-e78f-4a02-9c62-b613fb7d9fe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a67efdb6af84cfb92df1be6d8345ea9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:43.743192 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4136f-9e2d-4099-940d-98644ce32de9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:43.744957 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4136f-9e2d-4099-940d-98644ce32de9 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:43.748415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-55d55a6a-e78f-4a02-9c62-b613fb7d9fe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a67efdb6af84cfb92df1be6d8345ea9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:43.749114 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-55d55a6a-e78f-4a02-9c62-b613fb7d9fe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a67efdb6af84cfb92df1be6d8345ea9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:43.755764 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1421/2836] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:43.756715 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-55d55a6a-e78f-4a02-9c62-b613fb7d9fe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a67efdb6af84cfb92df1be6d8345ea9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:43.763461 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1416/2837] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:03:43 2022] DELETE /identity/v3/users/7a67efdb6af84cfb92df1be6d8345ea9 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:03:43.947887 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4136f-9e2d-4099-940d-98644ce32de9 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:43.958606 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4136f-9e2d-4099-940d-98644ce32de9 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:44.023041 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984290d0-ed22-4ffd-b290-c3d227615dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:44.023588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984290d0-ed22-4ffd-b290-c3d227615dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:44.023860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984290d0-ed22-4ffd-b290-c3d227615dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:44.024560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984290d0-ed22-4ffd-b290-c3d227615dfd admin admin] PATH_INFO: `/v3/projects/b7f7a616ffc84d23af3691a2672a6829` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:44.025281 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984290d0-ed22-4ffd-b290-c3d227615dfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7f7a616ffc84d23af3691a2672a6829)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:44.028780 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984290d0-ed22-4ffd-b290-c3d227615dfd admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:44.036319 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d55a6a-e78f-4a02-9c62-b613fb7d9fe5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:44.040206 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-55d55a6a-e78f-4a02-9c62-b613fb7d9fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:44.057640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d55a6a-e78f-4a02-9c62-b613fb7d9fe5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:44.067115 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9951c-c2a7-44fb-80ee-21c3cc6415c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:44.067928 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9951c-c2a7-44fb-80ee-21c3cc6415c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:44.068406 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9951c-c2a7-44fb-80ee-21c3cc6415c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:44.069200 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9951c-c2a7-44fb-80ee-21c3cc6415c5 admin admin] PATH_INFO: `/v3/projects/5823101cc155404fbaa66c3b9c191d3d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:44.070634 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9951c-c2a7-44fb-80ee-21c3cc6415c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5823101cc155404fbaa66c3b9c191d3d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:44.072597 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9951c-c2a7-44fb-80ee-21c3cc6415c5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:44.086113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-984290d0-ed22-4ffd-b290-c3d227615dfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7f7a616ffc84d23af3691a2672a6829', 'request_id': 'req-984290d0-ed22-4ffd-b290-c3d227615dfd'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:44.092317 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-984290d0-ed22-4ffd-b290-c3d227615dfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7f7a616ffc84d23af3691a2672a6829', 'request_id': 'req-984290d0-ed22-4ffd-b290-c3d227615dfd'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:44.099157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-984290d0-ed22-4ffd-b290-c3d227615dfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7f7a616ffc84d23af3691a2672a6829', 'request_id': 'req-984290d0-ed22-4ffd-b290-c3d227615dfd'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:44.113048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-984290d0-ed22-4ffd-b290-c3d227615dfd admin admin] The token cache is being invalidate because project b7f7a616ffc84d23af3691a2672a6829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:44.113331 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-984290d0-ed22-4ffd-b290-c3d227615dfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:44.113959 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-984290d0-ed22-4ffd-b290-c3d227615dfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:44.115497 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1422/2838] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:43 2022] DELETE /identity/v3/projects/b7f7a616ffc84d23af3691a2672a6829 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:44.120292 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee9de9-0545-4084-918c-377f543a6ac0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:44.120573 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee9de9-0545-4084-918c-377f543a6ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:44.121378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee9de9-0545-4084-918c-377f543a6ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:44.124917 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-65c9951c-c2a7-44fb-80ee-21c3cc6415c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5823101cc155404fbaa66c3b9c191d3d', 'request_id': 'req-65c9951c-c2a7-44fb-80ee-21c3cc6415c5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:44.129580 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=73,keystone:DELETE=25,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:44.132227 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-65c9951c-c2a7-44fb-80ee-21c3cc6415c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5823101cc155404fbaa66c3b9c191d3d', 'request_id': 'req-65c9951c-c2a7-44fb-80ee-21c3cc6415c5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:44.141129 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-65c9951c-c2a7-44fb-80ee-21c3cc6415c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5823101cc155404fbaa66c3b9c191d3d', 'request_id': 'req-65c9951c-c2a7-44fb-80ee-21c3cc6415c5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:44.142759 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-34ee9de9-0545-4084-918c-377f543a6ac0 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:44.144048 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-34ee9de9-0545-4084-918c-377f543a6ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:44.154509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-65c9951c-c2a7-44fb-80ee-21c3cc6415c5 admin admin] The token cache is being invalidate because project 5823101cc155404fbaa66c3b9c191d3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:44.154798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-65c9951c-c2a7-44fb-80ee-21c3cc6415c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:44.155417 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-65c9951c-c2a7-44fb-80ee-21c3cc6415c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:44.156911 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1417/2839] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:44 2022] DELETE /identity/v3/projects/5823101cc155404fbaa66c3b9c191d3d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:03:44.212025 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1423/2840] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:44.222303 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1049c-c884-4519-9fa5-14b3fdb51105 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:44.222660 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1049c-c884-4519-9fa5-14b3fdb51105 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:44.223173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1049c-c884-4519-9fa5-14b3fdb51105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:44.249747 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d1f1049c-c884-4519-9fa5-14b3fdb51105 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:44.251151 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d1f1049c-c884-4519-9fa5-14b3fdb51105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:44.259565 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1418/2841] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:44.263923 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee9de9-0545-4084-918c-377f543a6ac0 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:44.268947 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:44.275765 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee9de9-0545-4084-918c-377f543a6ac0 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:44.281563 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265bbd95-8da6-4553-b9ad-be7c070dacb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:44.282375 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265bbd95-8da6-4553-b9ad-be7c070dacb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:44.282653 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265bbd95-8da6-4553-b9ad-be7c070dacb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:44.282949 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265bbd95-8da6-4553-b9ad-be7c070dacb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:44.283647 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265bbd95-8da6-4553-b9ad-be7c070dacb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:03:44.283949 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265bbd95-8da6-4553-b9ad-be7c070dacb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:44.285595 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265bbd95-8da6-4553-b9ad-be7c070dacb8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:44.298665 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1424/2842] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:03:44.303176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1049c-c884-4519-9fa5-14b3fdb51105 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:44.313933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1049c-c884-4519-9fa5-14b3fdb51105 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:44.318775 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416ed6b-7e7d-4e00-a2a1-e83d5b00194b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:44.319410 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416ed6b-7e7d-4e00-a2a1-e83d5b00194b admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:44.319687 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416ed6b-7e7d-4e00-a2a1-e83d5b00194b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:44.319958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416ed6b-7e7d-4e00-a2a1-e83d5b00194b admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:44.320642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416ed6b-7e7d-4e00-a2a1-e83d5b00194b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:44.322573 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416ed6b-7e7d-4e00-a2a1-e83d5b00194b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:44.351582 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1419/2843] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:03:44 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:03:44.356411 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265bbd95-8da6-4553-b9ad-be7c070dacb8 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:44.367071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265bbd95-8da6-4553-b9ad-be7c070dacb8 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:44.437603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb86ff-2472-47c6-9fea-947ecc07e47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:44.438203 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb86ff-2472-47c6-9fea-947ecc07e47d admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:44.438476 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb86ff-2472-47c6-9fea-947ecc07e47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:44.438739 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb86ff-2472-47c6-9fea-947ecc07e47d admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:44.439363 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb86ff-2472-47c6-9fea-947ecc07e47d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:44.441282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb86ff-2472-47c6-9fea-947ecc07e47d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:44.461560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1425/2844] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:03:44 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:03:44.466048 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416ed6b-7e7d-4e00-a2a1-e83d5b00194b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:44.476019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416ed6b-7e7d-4e00-a2a1-e83d5b00194b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:44.481313 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a7d63-a16a-4472-bda3-77d0d6ef1473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:44.481893 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a7d63-a16a-4472-bda3-77d0d6ef1473 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:44.482168 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a7d63-a16a-4472-bda3-77d0d6ef1473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:44.482462 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a7d63-a16a-4472-bda3-77d0d6ef1473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:44.483086 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a7d63-a16a-4472-bda3-77d0d6ef1473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:44.484689 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a7d63-a16a-4472-bda3-77d0d6ef1473 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:44.494285 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1420/2845] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:03:44.498713 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb86ff-2472-47c6-9fea-947ecc07e47d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:44.508731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb86ff-2472-47c6-9fea-947ecc07e47d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:44.513549 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e9fd2-0642-48ec-8e0b-acf7d0c16d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:44.514138 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e9fd2-0642-48ec-8e0b-acf7d0c16d8f admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:44.514417 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e9fd2-0642-48ec-8e0b-acf7d0c16d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:44.514688 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e9fd2-0642-48ec-8e0b-acf7d0c16d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:44.515323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e9fd2-0642-48ec-8e0b-acf7d0c16d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:44.516938 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e9fd2-0642-48ec-8e0b-acf7d0c16d8f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:44.527250 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1426/2846] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:44.531227 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a7d63-a16a-4472-bda3-77d0d6ef1473 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:44.540933 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a7d63-a16a-4472-bda3-77d0d6ef1473 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:44.546027 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5572bf-0fc5-47d0-9c56-4416a49b282c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:44.546662 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5572bf-0fc5-47d0-9c56-4416a49b282c admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:44.546938 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5572bf-0fc5-47d0-9c56-4416a49b282c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:44.547207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5572bf-0fc5-47d0-9c56-4416a49b282c admin admin] PATH_INFO: `/v3/projects/0f785182c7d64af5ab5d37a4b95275b9/users/aef73a49de304706a59335eff37debf0/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:44.562473 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5572bf-0fc5-47d0-9c56-4416a49b282c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f785182c7d64af5ab5d37a4b95275b9, user_id=aef73a49de304706a59335eff37debf0, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:44.565165 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5572bf-0fc5-47d0-9c56-4416a49b282c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:44.578224 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1421/2847] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:03:44 2022] PUT /identity/v3/projects/0f785182c7d64af5ab5d37a4b95275b9/users/aef73a49de304706a59335eff37debf0/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:03:44.583162 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f68b2-7d5e-41b1-a9d5-b76d1b872720 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:44.583444 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f68b2-7d5e-41b1-a9d5-b76d1b872720 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:44.583715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f68b2-7d5e-41b1-a9d5-b76d1b872720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:44.603121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-444f68b2-7d5e-41b1-a9d5-b76d1b872720 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:44.604367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-444f68b2-7d5e-41b1-a9d5-b76d1b872720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:44.662243 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1427/2848] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:44.672358 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5572bf-0fc5-47d0-9c56-4416a49b282c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:44.674037 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0b5572bf-0fc5-47d0-9c56-4416a49b282c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:44.728630 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5572bf-0fc5-47d0-9c56-4416a49b282c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:44.732966 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65066f01-0ff4-4fd5-871e-dbac08d37e18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:44.733558 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65066f01-0ff4-4fd5-871e-dbac08d37e18 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:44.733830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65066f01-0ff4-4fd5-871e-dbac08d37e18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:44.734094 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65066f01-0ff4-4fd5-871e-dbac08d37e18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:44.742771 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65066f01-0ff4-4fd5-871e-dbac08d37e18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:44.744198 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65066f01-0ff4-4fd5-871e-dbac08d37e18 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:44.753698 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1422/2849] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:45.371429 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4e70b-f74e-40e7-a6fa-41572c87f308 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:45.371724 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4e70b-f74e-40e7-a6fa-41572c87f308 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:45.372192 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4e70b-f74e-40e7-a6fa-41572c87f308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:45.424188 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-0ce4e70b-f74e-40e7-a6fa-41572c87f308 None None] MFA Rules not processed for user `aef73a49de304706a59335eff37debf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:45.425412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0ce4e70b-f74e-40e7-a6fa-41572c87f308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:45.491055 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:45.498670 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1428/2850] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:45.509429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65066f01-0ff4-4fd5-871e-dbac08d37e18 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:45.510953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-65066f01-0ff4-4fd5-871e-dbac08d37e18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:45.570093 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65066f01-0ff4-4fd5-871e-dbac08d37e18 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:45.573646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558aea7f-e759-4834-9979-245a94db15f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:45.574877 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aea7f-e759-4834-9979-245a94db15f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:45.575339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aea7f-e759-4834-9979-245a94db15f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:45.575614 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aea7f-e759-4834-9979-245a94db15f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:45.584142 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558aea7f-e759-4834-9979-245a94db15f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:45.585562 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558aea7f-e759-4834-9979-245a94db15f9 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:45.594372 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1423/2851] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:46.960602 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e895c-6c91-401a-83e6-49671a922413 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:46.961092 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e895c-6c91-401a-83e6-49671a922413 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:46.961192 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e895c-6c91-401a-83e6-49671a922413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:46.980731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d52e895c-6c91-401a-83e6-49671a922413 None None] MFA Rules not processed for user `aef73a49de304706a59335eff37debf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:46.982054 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d52e895c-6c91-401a-83e6-49671a922413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:46.989134 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1429/2852] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:49.938324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558aea7f-e759-4834-9979-245a94db15f9 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:49.939706 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-558aea7f-e759-4834-9979-245a94db15f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:49.954200 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558aea7f-e759-4834-9979-245a94db15f9 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:49.959101 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedeed1-5146-4343-a3bf-d55a13d87f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:49.959716 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedeed1-5146-4343-a3bf-d55a13d87f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:49.960334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedeed1-5146-4343-a3bf-d55a13d87f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:49.960334 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedeed1-5146-4343-a3bf-d55a13d87f7e admin admin] PATH_INFO: `/v3/users/feb497c6a7574866b0c6c4196a5500b6` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:49.961395 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eedeed1-5146-4343-a3bf-d55a13d87f7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb497c6a7574866b0c6c4196a5500b6)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:49.963272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eedeed1-5146-4343-a3bf-d55a13d87f7e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:49.985262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:50.013072 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4eedeed1-5146-4343-a3bf-d55a13d87f7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb497c6a7574866b0c6c4196a5500b6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:50.025212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4eedeed1-5146-4343-a3bf-d55a13d87f7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb497c6a7574866b0c6c4196a5500b6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:50.033208 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4eedeed1-5146-4343-a3bf-d55a13d87f7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb497c6a7574866b0c6c4196a5500b6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:50.033840 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4eedeed1-5146-4343-a3bf-d55a13d87f7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb497c6a7574866b0c6c4196a5500b6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:50.040779 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4eedeed1-5146-4343-a3bf-d55a13d87f7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb497c6a7574866b0c6c4196a5500b6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:50.047915 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1424/2853] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:03:49 2022] DELETE /identity/v3/users/feb497c6a7574866b0c6c4196a5500b6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:50.052885 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b08021-3b45-4962-8b9f-f4f51767a321 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:50.053385 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b08021-3b45-4962-8b9f-f4f51767a321 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:50.053487 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b08021-3b45-4962-8b9f-f4f51767a321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:50.076390 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e8b08021-3b45-4962-8b9f-f4f51767a321 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:50.078043 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e8b08021-3b45-4962-8b9f-f4f51767a321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:50.149547 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1430/2854] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:03:50.159667 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedeed1-5146-4343-a3bf-d55a13d87f7e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:50.161100 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4eedeed1-5146-4343-a3bf-d55a13d87f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:50.220631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedeed1-5146-4343-a3bf-d55a13d87f7e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:50.224380 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807972e5-4d8b-43a5-9ba5-b712a243668a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:50.224982 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807972e5-4d8b-43a5-9ba5-b712a243668a service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:50.225261 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807972e5-4d8b-43a5-9ba5-b712a243668a service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:50.225532 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807972e5-4d8b-43a5-9ba5-b712a243668a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:50.235307 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807972e5-4d8b-43a5-9ba5-b712a243668a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:50.236759 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807972e5-4d8b-43a5-9ba5-b712a243668a service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:50.247415 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1425/2855] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:50.519747 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b08021-3b45-4962-8b9f-f4f51767a321 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:50.521287 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e8b08021-3b45-4962-8b9f-f4f51767a321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:50.533727 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b08021-3b45-4962-8b9f-f4f51767a321 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:50.539401 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830eb67-5908-4f04-8f61-4c72aa26aa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:50.540183 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830eb67-5908-4f04-8f61-4c72aa26aa8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:50.540541 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830eb67-5908-4f04-8f61-4c72aa26aa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:50.540906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830eb67-5908-4f04-8f61-4c72aa26aa8e admin admin] PATH_INFO: `/v3/projects/aa567ac48691480c908f6c69b0c8dfdb` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:50.542136 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830eb67-5908-4f04-8f61-4c72aa26aa8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa567ac48691480c908f6c69b0c8dfdb)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:50.544691 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830eb67-5908-4f04-8f61-4c72aa26aa8e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:50.594733 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d830eb67-5908-4f04-8f61-4c72aa26aa8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa567ac48691480c908f6c69b0c8dfdb', 'request_id': 'req-d830eb67-5908-4f04-8f61-4c72aa26aa8e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:50.600311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d830eb67-5908-4f04-8f61-4c72aa26aa8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa567ac48691480c908f6c69b0c8dfdb', 'request_id': 'req-d830eb67-5908-4f04-8f61-4c72aa26aa8e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:50.607360 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d830eb67-5908-4f04-8f61-4c72aa26aa8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa567ac48691480c908f6c69b0c8dfdb', 'request_id': 'req-d830eb67-5908-4f04-8f61-4c72aa26aa8e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:50.619173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d830eb67-5908-4f04-8f61-4c72aa26aa8e admin admin] The token cache is being invalidate because project aa567ac48691480c908f6c69b0c8dfdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:03:50.619463 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d830eb67-5908-4f04-8f61-4c72aa26aa8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:50.620108 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d830eb67-5908-4f04-8f61-4c72aa26aa8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:03:50.622144 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1431/2856] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:03:50 2022] DELETE /identity/v3/projects/aa567ac48691480c908f6c69b0c8dfdb => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:50.626997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf173706-c87b-45c0-8a5c-3d60c4432d41 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:50.627351 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf173706-c87b-45c0-8a5c-3d60c4432d41 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:50.627685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf173706-c87b-45c0-8a5c-3d60c4432d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:50.648766 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-bf173706-c87b-45c0-8a5c-3d60c4432d41 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:50.649963 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bf173706-c87b-45c0-8a5c-3d60c4432d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:50.707438 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1426/2857] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:03:50.716599 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d2320-c741-49f5-b04b-362368d31de1 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:50.716868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d2320-c741-49f5-b04b-362368d31de1 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:50.717133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d2320-c741-49f5-b04b-362368d31de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:50.736522 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-364d2320-c741-49f5-b04b-362368d31de1 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:50.737704 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-364d2320-c741-49f5-b04b-362368d31de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:50.745295 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1432/2858] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:50.750163 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf173706-c87b-45c0-8a5c-3d60c4432d41 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:50.759771 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf173706-c87b-45c0-8a5c-3d60c4432d41 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:50.765313 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafc464-8ce0-4fcd-8bc1-f69e794e03ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:50.765905 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafc464-8ce0-4fcd-8bc1-f69e794e03ea admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:50.766179 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafc464-8ce0-4fcd-8bc1-f69e794e03ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:50.766469 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafc464-8ce0-4fcd-8bc1-f69e794e03ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:50.767106 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafc464-8ce0-4fcd-8bc1-f69e794e03ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:03:50.767428 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafc464-8ce0-4fcd-8bc1-f69e794e03ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:50.769058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafc464-8ce0-4fcd-8bc1-f69e794e03ea admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:50.780633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1427/2859] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:03:50.784807 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d2320-c741-49f5-b04b-362368d31de1 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:50.794258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d2320-c741-49f5-b04b-362368d31de1 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:50.799168 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b7a48-d5d0-4197-9308-c38152150b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:50.799761 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b7a48-d5d0-4197-9308-c38152150b7f admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:50.800033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b7a48-d5d0-4197-9308-c38152150b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:50.800335 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b7a48-d5d0-4197-9308-c38152150b7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:50.800958 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b7a48-d5d0-4197-9308-c38152150b7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:50.802837 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b7a48-d5d0-4197-9308-c38152150b7f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:50.833685 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1433/2860] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:03:50 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:03:50.837849 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafc464-8ce0-4fcd-8bc1-f69e794e03ea admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:50.847581 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafc464-8ce0-4fcd-8bc1-f69e794e03ea admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:50.872699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:50.905564 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37adea0-b8b8-436b-8150-704a101155b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:50.906196 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37adea0-b8b8-436b-8150-704a101155b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:50.906475 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37adea0-b8b8-436b-8150-704a101155b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:50.906748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37adea0-b8b8-436b-8150-704a101155b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:50.907413 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37adea0-b8b8-436b-8150-704a101155b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:50.909409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37adea0-b8b8-436b-8150-704a101155b4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:50.928985 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1428/2861] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:03:50 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:03:50.933755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b7a48-d5d0-4197-9308-c38152150b7f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:50.944389 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b7a48-d5d0-4197-9308-c38152150b7f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:50.949109 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e765a19-59f3-4064-b574-91da1a7f24dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:50.949707 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e765a19-59f3-4064-b574-91da1a7f24dc admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:50.949982 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e765a19-59f3-4064-b574-91da1a7f24dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:50.950253 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e765a19-59f3-4064-b574-91da1a7f24dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:50.950884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e765a19-59f3-4064-b574-91da1a7f24dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:50.952511 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e765a19-59f3-4064-b574-91da1a7f24dc admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:50.962319 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1434/2862] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:50.966222 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37adea0-b8b8-436b-8150-704a101155b4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:50.976068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37adea0-b8b8-436b-8150-704a101155b4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:50.981259 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cde7d3-e570-4e62-9045-990328ea11e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:50.981841 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cde7d3-e570-4e62-9045-990328ea11e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:50.982116 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cde7d3-e570-4e62-9045-990328ea11e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:50.982385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cde7d3-e570-4e62-9045-990328ea11e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:50.983013 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cde7d3-e570-4e62-9045-990328ea11e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:50.984624 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cde7d3-e570-4e62-9045-990328ea11e0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:50.995308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1429/2863] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:03:51.000611 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e765a19-59f3-4064-b574-91da1a7f24dc admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:51.011537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e765a19-59f3-4064-b574-91da1a7f24dc admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:51.016794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f6dc8-55ea-49b3-bf2c-c76ba9fb3124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:51.017781 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f6dc8-55ea-49b3-bf2c-c76ba9fb3124 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:51.018143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f6dc8-55ea-49b3-bf2c-c76ba9fb3124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:51.018587 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f6dc8-55ea-49b3-bf2c-c76ba9fb3124 admin admin] PATH_INFO: `/v3/projects/db894a1c343942e3be94240770b3a265/users/49e563c07fef402aa03183b3c6784c9e/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:51.034977 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f6dc8-55ea-49b3-bf2c-c76ba9fb3124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db894a1c343942e3be94240770b3a265, user_id=49e563c07fef402aa03183b3c6784c9e, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:51.037693 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f6dc8-55ea-49b3-bf2c-c76ba9fb3124 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:51.052697 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1435/2864] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:03:50 2022] PUT /identity/v3/projects/db894a1c343942e3be94240770b3a265/users/49e563c07fef402aa03183b3c6784c9e/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:03:51.064216 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd0897-2e9e-4d25-8be7-03cd605ed6b9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:51.064606 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd0897-2e9e-4d25-8be7-03cd605ed6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:51.064860 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd0897-2e9e-4d25-8be7-03cd605ed6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:51.096409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1bbd0897-2e9e-4d25-8be7-03cd605ed6b9 None None] MFA Rules not processed for user `49e563c07fef402aa03183b3c6784c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:51.097709 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1bbd0897-2e9e-4d25-8be7-03cd605ed6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:51.164018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1430/2865] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:03:51.175019 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f6dc8-55ea-49b3-bf2c-c76ba9fb3124 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:03:51.176763 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-222f6dc8-55ea-49b3-bf2c-c76ba9fb3124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:51.191070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:03:51.235677 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f6dc8-55ea-49b3-bf2c-c76ba9fb3124 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:03:51.239283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe299b-7666-4370-9c01-9d8e24cf85c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:03:51.239878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe299b-7666-4370-9c01-9d8e24cf85c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:51.240185 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe299b-7666-4370-9c01-9d8e24cf85c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:51.240464 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe299b-7666-4370-9c01-9d8e24cf85c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:51.249350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe299b-7666-4370-9c01-9d8e24cf85c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:03:51.250788 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe299b-7666-4370-9c01-9d8e24cf85c6 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:03:51.259456 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1436/2866] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:03:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:03:52.633335 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eee491-4004-4207-8dfa-b8880b41e19e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:03:52.633853 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eee491-4004-4207-8dfa-b8880b41e19e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:03:52.633997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eee491-4004-4207-8dfa-b8880b41e19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:03:52.652866 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c3eee491-4004-4207-8dfa-b8880b41e19e None None] MFA Rules not processed for user `49e563c07fef402aa03183b3c6784c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:03:52.654055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c3eee491-4004-4207-8dfa-b8880b41e19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:03:52.660776 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1431/2867] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:03:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:04:01.255581 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=21 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:04:02.647449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=56,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:04:07.671282 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe299b-7666-4370-9c01-9d8e24cf85c6 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:07.672712 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-15fe299b-7666-4370-9c01-9d8e24cf85c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:07.766052 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe299b-7666-4370-9c01-9d8e24cf85c6 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:07.771263 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57fc94-1c85-4bb6-968f-67c3072bab44 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:07.771889 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57fc94-1c85-4bb6-968f-67c3072bab44 service placement] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:07.772190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57fc94-1c85-4bb6-968f-67c3072bab44 service placement] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:07.772492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57fc94-1c85-4bb6-968f-67c3072bab44 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:07.774235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8e57fc94-1c85-4bb6-968f-67c3072bab44 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:07.786766 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57fc94-1c85-4bb6-968f-67c3072bab44 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:07.790161 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57fc94-1c85-4bb6-968f-67c3072bab44 service placement] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:07.800604 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1437/2868] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:04:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:04:09.038038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc63f9-c82d-491d-8731-6ec54a5f58c0 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:09.038530 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc63f9-c82d-491d-8731-6ec54a5f58c0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:09.038641 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc63f9-c82d-491d-8731-6ec54a5f58c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:09.057588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-12bc63f9-c82d-491d-8731-6ec54a5f58c0 None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:04:09.059419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-12bc63f9-c82d-491d-8731-6ec54a5f58c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:09.112516 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1432/2869] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Sep 7 08:04:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:04:09.124868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57fc94-1c85-4bb6-968f-67c3072bab44 service placement] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:09.126289 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8e57fc94-1c85-4bb6-968f-67c3072bab44 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:09.196651 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57fc94-1c85-4bb6-968f-67c3072bab44 service placement] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:09.201634 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ae7d8-240c-487b-be74-f2ee8aa8165b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:09.202227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ae7d8-240c-487b-be74-f2ee8aa8165b service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:09.202529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ae7d8-240c-487b-be74-f2ee8aa8165b service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:09.202794 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ae7d8-240c-487b-be74-f2ee8aa8165b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:09.210486 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ae7d8-240c-487b-be74-f2ee8aa8165b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:09.212051 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ae7d8-240c-487b-be74-f2ee8aa8165b service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:09.219928 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1438/2870] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 08:04:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 08:04:09.729623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fda8c-4ce5-489f-ad4c-0b1c2980de84 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:09.729912 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fda8c-4ce5-489f-ad4c-0b1c2980de84 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:09.730196 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fda8c-4ce5-489f-ad4c-0b1c2980de84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:09.749437 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-932fda8c-4ce5-489f-ad4c-0b1c2980de84 None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:04:09.750633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-932fda8c-4ce5-489f-ad4c-0b1c2980de84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:09.758066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1433/2871] 10.222.0.26 () {58 vars in 967 bytes} [Wed Sep 7 08:04:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:04:09.771084 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ae7d8-240c-487b-be74-f2ee8aa8165b service swift] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:09.782388 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ae7d8-240c-487b-be74-f2ee8aa8165b service swift] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:09.787441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7dbb37-7546-4c15-9108-12b46c9df645 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:09.788027 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7dbb37-7546-4c15-9108-12b46c9df645 service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:09.788338 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7dbb37-7546-4c15-9108-12b46c9df645 service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:09.788609 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7dbb37-7546-4c15-9108-12b46c9df645 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:09.796618 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7dbb37-7546-4c15-9108-12b46c9df645 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:09.798185 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7dbb37-7546-4c15-9108-12b46c9df645 service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:09.806025 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1439/2872] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 08:04:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 08:04:10.026856 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334e24e-7f50-4532-a1ed-95577adf21bc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:10.027152 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334e24e-7f50-4532-a1ed-95577adf21bc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:10.027959 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334e24e-7f50-4532-a1ed-95577adf21bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:10.046661 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b334e24e-7f50-4532-a1ed-95577adf21bc None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:04:10.047832 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b334e24e-7f50-4532-a1ed-95577adf21bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:10.054752 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1434/2873] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Sep 7 08:04:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:04:10.067396 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7dbb37-7546-4c15-9108-12b46c9df645 service swift] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:10.077906 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7dbb37-7546-4c15-9108-12b46c9df645 service swift] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:10.082823 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada87d2-d9b7-4fc9-b342-67ef1bfb5cc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:10.083409 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada87d2-d9b7-4fc9-b342-67ef1bfb5cc3 service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:10.083683 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada87d2-d9b7-4fc9-b342-67ef1bfb5cc3 service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:10.083953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada87d2-d9b7-4fc9-b342-67ef1bfb5cc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:10.092470 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada87d2-d9b7-4fc9-b342-67ef1bfb5cc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:10.094029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada87d2-d9b7-4fc9-b342-67ef1bfb5cc3 service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:10.101860 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1440/2874] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 08:04:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 08:04:10.418475 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecafb2d-1f2e-415b-93c8-a8d46f810c62 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:10.418761 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecafb2d-1f2e-415b-93c8-a8d46f810c62 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:10.419032 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecafb2d-1f2e-415b-93c8-a8d46f810c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:10.438811 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8ecafb2d-1f2e-415b-93c8-a8d46f810c62 None None] MFA Rules not processed for user `617f6479b90945d3b5a63325839afb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:04:10.440008 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8ecafb2d-1f2e-415b-93c8-a8d46f810c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:10.447068 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1435/2875] 10.222.0.26 () {58 vars in 967 bytes} [Wed Sep 7 08:04:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 08:04:10.459535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada87d2-d9b7-4fc9-b342-67ef1bfb5cc3 service swift] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:10.470384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada87d2-d9b7-4fc9-b342-67ef1bfb5cc3 service swift] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:10.475410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1643678-899a-4c9f-9e12-61507f0ebfda service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff3bf8da732489b8b3a8fbe5aa22f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:10.476101 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1643678-899a-4c9f-9e12-61507f0ebfda service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:10.476411 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1643678-899a-4c9f-9e12-61507f0ebfda service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:10.476686 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1643678-899a-4c9f-9e12-61507f0ebfda service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:10.485471 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1643678-899a-4c9f-9e12-61507f0ebfda service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:10.487070 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1643678-899a-4c9f-9e12-61507f0ebfda service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:10.494822 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1441/2876] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 7 08:04:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 08:04:12.591035 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecafb2d-1f2e-415b-93c8-a8d46f810c62 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:12.653296 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecafb2d-1f2e-415b-93c8-a8d46f810c62 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:12.657836 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72e75a-52c3-4bbe-ae4b-ac440146937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:12.658459 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72e75a-52c3-4bbe-ae4b-ac440146937c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:12.658735 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72e75a-52c3-4bbe-ae4b-ac440146937c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:12.659009 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72e75a-52c3-4bbe-ae4b-ac440146937c admin admin] PATH_INFO: `/v3/users/49e563c07fef402aa03183b3c6784c9e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:12.660153 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72e75a-52c3-4bbe-ae4b-ac440146937c admin admin] RBAC: Authorizing `identity:delete_user(user_id=49e563c07fef402aa03183b3c6784c9e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:12.662055 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72e75a-52c3-4bbe-ae4b-ac440146937c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:12.710331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cd72e75a-52c3-4bbe-ae4b-ac440146937c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49e563c07fef402aa03183b3c6784c9e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:12.722884 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cd72e75a-52c3-4bbe-ae4b-ac440146937c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49e563c07fef402aa03183b3c6784c9e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:12.729424 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cd72e75a-52c3-4bbe-ae4b-ac440146937c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49e563c07fef402aa03183b3c6784c9e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:12.729897 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cd72e75a-52c3-4bbe-ae4b-ac440146937c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49e563c07fef402aa03183b3c6784c9e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:12.736695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cd72e75a-52c3-4bbe-ae4b-ac440146937c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49e563c07fef402aa03183b3c6784c9e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:12.742575 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1436/2877] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:04:12 2022] DELETE /identity/v3/users/49e563c07fef402aa03183b3c6784c9e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:04:12.747171 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106af7fa-33e5-453b-a5cc-1b67d6c6a605 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:12.747674 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106af7fa-33e5-453b-a5cc-1b67d6c6a605 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:12.747812 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106af7fa-33e5-453b-a5cc-1b67d6c6a605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:12.767017 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-106af7fa-33e5-453b-a5cc-1b67d6c6a605 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:04:12.768236 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-106af7fa-33e5-453b-a5cc-1b67d6c6a605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:12.780925 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:04:12.826726 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1442/2878] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:04:12.836105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72e75a-52c3-4bbe-ae4b-ac440146937c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:12.837737 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cd72e75a-52c3-4bbe-ae4b-ac440146937c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:12.845446 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:04:12.907311 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72e75a-52c3-4bbe-ae4b-ac440146937c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:12.912330 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1400d7-2dc3-4ed1-955e-538023c69cb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:12.912932 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1400d7-2dc3-4ed1-955e-538023c69cb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:12.913235 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1400d7-2dc3-4ed1-955e-538023c69cb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:12.913503 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1400d7-2dc3-4ed1-955e-538023c69cb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:12.923215 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1400d7-2dc3-4ed1-955e-538023c69cb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:12.924293 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1400d7-2dc3-4ed1-955e-538023c69cb6 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:12.933975 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1437/2879] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:04:13.112591 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106af7fa-33e5-453b-a5cc-1b67d6c6a605 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:13.115384 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-106af7fa-33e5-453b-a5cc-1b67d6c6a605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:13.135581 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106af7fa-33e5-453b-a5cc-1b67d6c6a605 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:13.141131 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be91f5-c559-4008-b488-3fa8eef8b2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:13.142126 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be91f5-c559-4008-b488-3fa8eef8b2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:13.143001 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be91f5-c559-4008-b488-3fa8eef8b2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:13.143433 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be91f5-c559-4008-b488-3fa8eef8b2b7 admin admin] PATH_INFO: `/v3/projects/db894a1c343942e3be94240770b3a265` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:13.144608 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be91f5-c559-4008-b488-3fa8eef8b2b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db894a1c343942e3be94240770b3a265)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:13.146690 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be91f5-c559-4008-b488-3fa8eef8b2b7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:13.198852 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d2be91f5-c559-4008-b488-3fa8eef8b2b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db894a1c343942e3be94240770b3a265', 'request_id': 'req-d2be91f5-c559-4008-b488-3fa8eef8b2b7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:13.205730 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d2be91f5-c559-4008-b488-3fa8eef8b2b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db894a1c343942e3be94240770b3a265', 'request_id': 'req-d2be91f5-c559-4008-b488-3fa8eef8b2b7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:13.212253 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d2be91f5-c559-4008-b488-3fa8eef8b2b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db894a1c343942e3be94240770b3a265', 'request_id': 'req-d2be91f5-c559-4008-b488-3fa8eef8b2b7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:13.225385 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d2be91f5-c559-4008-b488-3fa8eef8b2b7 admin admin] The token cache is being invalidate because project db894a1c343942e3be94240770b3a265 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:04:13.225907 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d2be91f5-c559-4008-b488-3fa8eef8b2b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:13.226509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d2be91f5-c559-4008-b488-3fa8eef8b2b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:13.228365 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1443/2880] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:04:13 2022] DELETE /identity/v3/projects/db894a1c343942e3be94240770b3a265 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:04:13.233465 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f8062-a324-4908-bec2-a2ebd65d8e58 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:13.236635 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f8062-a324-4908-bec2-a2ebd65d8e58 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:13.236635 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f8062-a324-4908-bec2-a2ebd65d8e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:13.259229 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-948f8062-a324-4908-bec2-a2ebd65d8e58 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:04:13.260484 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-948f8062-a324-4908-bec2-a2ebd65d8e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:13.323367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1438/2881] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:04:13.331439 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61069f7a-1736-4a5b-9d9c-52390bdf5c33 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:13.331723 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61069f7a-1736-4a5b-9d9c-52390bdf5c33 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:13.332023 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61069f7a-1736-4a5b-9d9c-52390bdf5c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:13.355136 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-61069f7a-1736-4a5b-9d9c-52390bdf5c33 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:04:13.356392 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-61069f7a-1736-4a5b-9d9c-52390bdf5c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:13.364294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1444/2882] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:04:13.368512 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f8062-a324-4908-bec2-a2ebd65d8e58 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:13.378854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f8062-a324-4908-bec2-a2ebd65d8e58 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:13.383770 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e59e0-7fa6-4cc0-9dcc-b2d0b4e67b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:13.384531 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e59e0-7fa6-4cc0-9dcc-b2d0b4e67b31 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:13.384808 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e59e0-7fa6-4cc0-9dcc-b2d0b4e67b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:13.385082 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e59e0-7fa6-4cc0-9dcc-b2d0b4e67b31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:13.385742 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e59e0-7fa6-4cc0-9dcc-b2d0b4e67b31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:04:13.386414 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e59e0-7fa6-4cc0-9dcc-b2d0b4e67b31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:13.388240 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e59e0-7fa6-4cc0-9dcc-b2d0b4e67b31 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:13.400954 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1439/2883] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:04:13.405469 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61069f7a-1736-4a5b-9d9c-52390bdf5c33 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:13.414979 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61069f7a-1736-4a5b-9d9c-52390bdf5c33 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:13.420715 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b49487-10fb-49d1-bb57-737207935b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:13.421312 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b49487-10fb-49d1-bb57-737207935b3a admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:13.421588 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b49487-10fb-49d1-bb57-737207935b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:13.421856 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b49487-10fb-49d1-bb57-737207935b3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:13.422484 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b49487-10fb-49d1-bb57-737207935b3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:13.424458 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b49487-10fb-49d1-bb57-737207935b3a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:13.453775 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1445/2884] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:04:13 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:04:13.458021 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e59e0-7fa6-4cc0-9dcc-b2d0b4e67b31 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:13.468061 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e59e0-7fa6-4cc0-9dcc-b2d0b4e67b31 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:13.525977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8c1a7-7783-4efe-a75e-c607ea26fbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:13.526509 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8c1a7-7783-4efe-a75e-c607ea26fbfb admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:13.526783 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8c1a7-7783-4efe-a75e-c607ea26fbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:13.527088 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8c1a7-7783-4efe-a75e-c607ea26fbfb admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:13.527886 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8c1a7-7783-4efe-a75e-c607ea26fbfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:13.529834 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8c1a7-7783-4efe-a75e-c607ea26fbfb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:13.551427 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1440/2885] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:04:13 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:04:13.555884 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b49487-10fb-49d1-bb57-737207935b3a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:13.567039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b49487-10fb-49d1-bb57-737207935b3a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:13.571722 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615c8fa-f53b-4059-b01a-de6515503e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:13.572395 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615c8fa-f53b-4059-b01a-de6515503e07 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:13.572714 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615c8fa-f53b-4059-b01a-de6515503e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:13.573008 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615c8fa-f53b-4059-b01a-de6515503e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:13.573646 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615c8fa-f53b-4059-b01a-de6515503e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:13.576273 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615c8fa-f53b-4059-b01a-de6515503e07 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:13.586466 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1446/2886] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:04:13.591079 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8c1a7-7783-4efe-a75e-c607ea26fbfb admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:13.600940 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8c1a7-7783-4efe-a75e-c607ea26fbfb admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:13.605565 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad99534-48ae-45ea-9016-1f44650b2bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:13.606144 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad99534-48ae-45ea-9016-1f44650b2bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:13.606422 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad99534-48ae-45ea-9016-1f44650b2bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:13.606693 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad99534-48ae-45ea-9016-1f44650b2bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:13.607322 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad99534-48ae-45ea-9016-1f44650b2bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:13.608960 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad99534-48ae-45ea-9016-1f44650b2bc5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:13.619193 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1441/2887] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:04:13.623354 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615c8fa-f53b-4059-b01a-de6515503e07 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:13.633741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615c8fa-f53b-4059-b01a-de6515503e07 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:13.639379 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bec24e-925d-4345-8828-f655ae572114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:13.640023 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bec24e-925d-4345-8828-f655ae572114 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:13.640332 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bec24e-925d-4345-8828-f655ae572114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:13.640603 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bec24e-925d-4345-8828-f655ae572114 admin admin] PATH_INFO: `/v3/projects/e56848c07a4b456d9f8113640d58aa13/users/6e806b6d7ddb4059bb88374497432552/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:13.656388 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bec24e-925d-4345-8828-f655ae572114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e56848c07a4b456d9f8113640d58aa13, user_id=6e806b6d7ddb4059bb88374497432552, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:13.659076 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bec24e-925d-4345-8828-f655ae572114 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:13.672473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1447/2888] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:04:13 2022] PUT /identity/v3/projects/e56848c07a4b456d9f8113640d58aa13/users/6e806b6d7ddb4059bb88374497432552/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:04:13.742113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dc323-71f6-490f-9f0d-70cccb15afa9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:13.742400 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dc323-71f6-490f-9f0d-70cccb15afa9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:13.742699 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dc323-71f6-490f-9f0d-70cccb15afa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:13.768554 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:04:13.772335 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4b3dc323-71f6-490f-9f0d-70cccb15afa9 None None] MFA Rules not processed for user `6e806b6d7ddb4059bb88374497432552`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:04:13.774192 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4b3dc323-71f6-490f-9f0d-70cccb15afa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:13.843078 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1442/2889] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:04:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:04:13.853792 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bec24e-925d-4345-8828-f655ae572114 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:13.855584 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-45bec24e-925d-4345-8828-f655ae572114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:13.899169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:04:13.914878 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bec24e-925d-4345-8828-f655ae572114 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:13.918596 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce99f5-ffeb-42ac-9b1c-110b4c4db222 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:13.919192 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce99f5-ffeb-42ac-9b1c-110b4c4db222 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:13.919463 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce99f5-ffeb-42ac-9b1c-110b4c4db222 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:13.919731 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce99f5-ffeb-42ac-9b1c-110b4c4db222 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:13.928137 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce99f5-ffeb-42ac-9b1c-110b4c4db222 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:13.929561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce99f5-ffeb-42ac-9b1c-110b4c4db222 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:13.938987 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1448/2890] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:04:13 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:04:23.840707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=26 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:04:23.934778 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=8 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:04:35.326345 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3dc323-71f6-490f-9f0d-70cccb15afa9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:35.388640 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3dc323-71f6-490f-9f0d-70cccb15afa9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:35.393157 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc96709-ce83-4f12-826c-2e185762d1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:35.393776 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc96709-ce83-4f12-826c-2e185762d1b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:35.394074 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc96709-ce83-4f12-826c-2e185762d1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:35.394331 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc96709-ce83-4f12-826c-2e185762d1b0 admin admin] PATH_INFO: `/v3/users/6e806b6d7ddb4059bb88374497432552` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:35.395447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc96709-ce83-4f12-826c-2e185762d1b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e806b6d7ddb4059bb88374497432552)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:35.397375 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc96709-ce83-4f12-826c-2e185762d1b0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:35.442777 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6cc96709-ce83-4f12-826c-2e185762d1b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e806b6d7ddb4059bb88374497432552'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:35.454705 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6cc96709-ce83-4f12-826c-2e185762d1b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e806b6d7ddb4059bb88374497432552'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:35.461207 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6cc96709-ce83-4f12-826c-2e185762d1b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e806b6d7ddb4059bb88374497432552'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:35.461832 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6cc96709-ce83-4f12-826c-2e185762d1b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e806b6d7ddb4059bb88374497432552'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:35.468798 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6cc96709-ce83-4f12-826c-2e185762d1b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e806b6d7ddb4059bb88374497432552'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:35.474769 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1443/2891] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:04:35 2022] DELETE /identity/v3/users/6e806b6d7ddb4059bb88374497432552 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:04:35.479336 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da826a7-d14b-4fc1-bd56-6a016c8ebff3 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:35.479798 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da826a7-d14b-4fc1-bd56-6a016c8ebff3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:35.480437 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da826a7-d14b-4fc1-bd56-6a016c8ebff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:35.499305 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3da826a7-d14b-4fc1-bd56-6a016c8ebff3 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:04:35.500519 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3da826a7-d14b-4fc1-bd56-6a016c8ebff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:35.556287 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1449/2892] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:04:35.566399 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc96709-ce83-4f12-826c-2e185762d1b0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:35.567823 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6cc96709-ce83-4f12-826c-2e185762d1b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:35.624388 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc96709-ce83-4f12-826c-2e185762d1b0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:35.628073 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d2afe-9d55-4ca4-b21d-5cf971fb5117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:35.628696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d2afe-9d55-4ca4-b21d-5cf971fb5117 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:35.628977 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d2afe-9d55-4ca4-b21d-5cf971fb5117 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:35.629247 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d2afe-9d55-4ca4-b21d-5cf971fb5117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:35.638315 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d2afe-9d55-4ca4-b21d-5cf971fb5117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:35.639729 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d2afe-9d55-4ca4-b21d-5cf971fb5117 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:35.649105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1444/2893] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:04:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:04:35.927529 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da826a7-d14b-4fc1-bd56-6a016c8ebff3 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:35.928970 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3da826a7-d14b-4fc1-bd56-6a016c8ebff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:35.941119 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da826a7-d14b-4fc1-bd56-6a016c8ebff3 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:35.945694 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c23c46-29f1-4f1b-9d3f-662cd764cbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:35.946321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c23c46-29f1-4f1b-9d3f-662cd764cbfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:35.946598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c23c46-29f1-4f1b-9d3f-662cd764cbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:35.946868 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c23c46-29f1-4f1b-9d3f-662cd764cbfe admin admin] PATH_INFO: `/v3/projects/e56848c07a4b456d9f8113640d58aa13` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:35.948241 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c23c46-29f1-4f1b-9d3f-662cd764cbfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e56848c07a4b456d9f8113640d58aa13)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:35.950831 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c23c46-29f1-4f1b-9d3f-662cd764cbfe admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:36.005438 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d0c23c46-29f1-4f1b-9d3f-662cd764cbfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e56848c07a4b456d9f8113640d58aa13', 'request_id': 'req-d0c23c46-29f1-4f1b-9d3f-662cd764cbfe'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:36.010886 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d0c23c46-29f1-4f1b-9d3f-662cd764cbfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e56848c07a4b456d9f8113640d58aa13', 'request_id': 'req-d0c23c46-29f1-4f1b-9d3f-662cd764cbfe'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:36.018513 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d0c23c46-29f1-4f1b-9d3f-662cd764cbfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e56848c07a4b456d9f8113640d58aa13', 'request_id': 'req-d0c23c46-29f1-4f1b-9d3f-662cd764cbfe'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:36.030629 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d0c23c46-29f1-4f1b-9d3f-662cd764cbfe admin admin] The token cache is being invalidate because project e56848c07a4b456d9f8113640d58aa13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:04:36.030917 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d0c23c46-29f1-4f1b-9d3f-662cd764cbfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:36.031533 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d0c23c46-29f1-4f1b-9d3f-662cd764cbfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:04:36.033046 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1450/2894] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:04:35 2022] DELETE /identity/v3/projects/e56848c07a4b456d9f8113640d58aa13 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:04:36.038058 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdf81f-f37a-4b64-8cf3-37d54ee1dfa1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:36.038367 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdf81f-f37a-4b64-8cf3-37d54ee1dfa1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:36.038618 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdf81f-f37a-4b64-8cf3-37d54ee1dfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:36.058033 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5dcdf81f-f37a-4b64-8cf3-37d54ee1dfa1 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:04:36.059529 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5dcdf81f-f37a-4b64-8cf3-37d54ee1dfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:36.078299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:04:36.121712 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1445/2895] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:04:36.129503 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8ef53-853e-45b9-a681-7aa467419976 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:36.129787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8ef53-853e-45b9-a681-7aa467419976 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:36.130065 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8ef53-853e-45b9-a681-7aa467419976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:36.150087 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e3e8ef53-853e-45b9-a681-7aa467419976 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:04:36.151296 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e3e8ef53-853e-45b9-a681-7aa467419976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:36.159678 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1451/2896] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:04:36.163822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcdf81f-f37a-4b64-8cf3-37d54ee1dfa1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:36.173443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcdf81f-f37a-4b64-8cf3-37d54ee1dfa1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:36.178113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880274d-4f75-4065-a9c0-369703deca83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:36.178706 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880274d-4f75-4065-a9c0-369703deca83 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:36.178988 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880274d-4f75-4065-a9c0-369703deca83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:36.179261 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880274d-4f75-4065-a9c0-369703deca83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:36.179891 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880274d-4f75-4065-a9c0-369703deca83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 08:04:36.180230 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880274d-4f75-4065-a9c0-369703deca83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:36.181794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880274d-4f75-4065-a9c0-369703deca83 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:36.193885 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1446/2897] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Sep 7 08:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 08:04:36.198051 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8ef53-853e-45b9-a681-7aa467419976 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:36.207960 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8ef53-853e-45b9-a681-7aa467419976 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:36.212632 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd551aa6-b339-47ae-86c4-336fbbf08585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:36.213254 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd551aa6-b339-47ae-86c4-336fbbf08585 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:36.213532 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd551aa6-b339-47ae-86c4-336fbbf08585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:36.213830 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd551aa6-b339-47ae-86c4-336fbbf08585 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:36.215253 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd551aa6-b339-47ae-86c4-336fbbf08585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:36.217181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd551aa6-b339-47ae-86c4-336fbbf08585 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:36.244502 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1452/2898] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:04:36 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:04:36.248691 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880274d-4f75-4065-a9c0-369703deca83 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:36.259148 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880274d-4f75-4065-a9c0-369703deca83 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:36.314066 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52150f35-8d20-41e9-89a7-ad0b8f0c8e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:36.314653 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52150f35-8d20-41e9-89a7-ad0b8f0c8e3c admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:36.314930 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52150f35-8d20-41e9-89a7-ad0b8f0c8e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:36.315204 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52150f35-8d20-41e9-89a7-ad0b8f0c8e3c admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:36.315828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52150f35-8d20-41e9-89a7-ad0b8f0c8e3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:36.317755 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52150f35-8d20-41e9-89a7-ad0b8f0c8e3c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:36.338716 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1447/2899] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:04:36 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:04:36.342782 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd551aa6-b339-47ae-86c4-336fbbf08585 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:36.353121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd551aa6-b339-47ae-86c4-336fbbf08585 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:36.357941 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e1d36-a8cf-4b61-bafd-2473c22636f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:36.358535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e1d36-a8cf-4b61-bafd-2473c22636f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:36.358814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e1d36-a8cf-4b61-bafd-2473c22636f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:36.359086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e1d36-a8cf-4b61-bafd-2473c22636f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:36.359717 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e1d36-a8cf-4b61-bafd-2473c22636f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:36.361350 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e1d36-a8cf-4b61-bafd-2473c22636f1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:36.371880 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1453/2900] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:04:36.375707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52150f35-8d20-41e9-89a7-ad0b8f0c8e3c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:36.385986 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52150f35-8d20-41e9-89a7-ad0b8f0c8e3c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:36.391368 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15c693-816d-4e70-88db-87970606db8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:36.391965 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15c693-816d-4e70-88db-87970606db8d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:36.392273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15c693-816d-4e70-88db-87970606db8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:36.392545 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15c693-816d-4e70-88db-87970606db8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:36.393184 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15c693-816d-4e70-88db-87970606db8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:36.394757 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15c693-816d-4e70-88db-87970606db8d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:36.405588 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1448/2901] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:04:36.409446 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e1d36-a8cf-4b61-bafd-2473c22636f1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:36.419463 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e1d36-a8cf-4b61-bafd-2473c22636f1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:36.424440 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40e2ba-54a3-4fb4-bc53-d01725f37cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:36.424825 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40e2ba-54a3-4fb4-bc53-d01725f37cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:36.425102 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40e2ba-54a3-4fb4-bc53-d01725f37cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:36.425373 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40e2ba-54a3-4fb4-bc53-d01725f37cbe admin admin] PATH_INFO: `/v3/projects/e2744f91a6284e0688af92a926c199d1/users/19af2d1084c14c3ea2df8369855aaed0/roles/6ebbed85905048a198e4261a3510dafc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:36.440806 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40e2ba-54a3-4fb4-bc53-d01725f37cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2744f91a6284e0688af92a926c199d1, user_id=19af2d1084c14c3ea2df8369855aaed0, role_id=6ebbed85905048a198e4261a3510dafc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:36.443421 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40e2ba-54a3-4fb4-bc53-d01725f37cbe admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:36.457006 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1454/2902] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:04:36 2022] PUT /identity/v3/projects/e2744f91a6284e0688af92a926c199d1/users/19af2d1084c14c3ea2df8369855aaed0/roles/6ebbed85905048a198e4261a3510dafc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:04:36.461987 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74155f-7a7d-4d1f-8f20-d30d2672edc5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:36.462272 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74155f-7a7d-4d1f-8f20-d30d2672edc5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:36.462540 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74155f-7a7d-4d1f-8f20-d30d2672edc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:36.482328 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9e74155f-7a7d-4d1f-8f20-d30d2672edc5 None None] MFA Rules not processed for user `984123d0eaef4929af01f39a46ebd51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:04:36.483517 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9e74155f-7a7d-4d1f-8f20-d30d2672edc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:36.539688 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1449/2903] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:04:36.549311 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40e2ba-54a3-4fb4-bc53-d01725f37cbe admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:36.551071 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ce40e2ba-54a3-4fb4-bc53-d01725f37cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:36.573288 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:04:36.609448 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40e2ba-54a3-4fb4-bc53-d01725f37cbe admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:36.613022 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc332767-cd1c-4769-8758-341e38838c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:36.613612 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc332767-cd1c-4769-8758-341e38838c5f service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:36.613897 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc332767-cd1c-4769-8758-341e38838c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:36.614167 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc332767-cd1c-4769-8758-341e38838c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:36.623765 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc332767-cd1c-4769-8758-341e38838c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:36.625209 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc332767-cd1c-4769-8758-341e38838c5f service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:36.634383 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1455/2904] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:04:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:04:40.687734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b4faa-af42-4a20-92f0-be3d99c4e37e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:40.688324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b4faa-af42-4a20-92f0-be3d99c4e37e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:40.688324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b4faa-af42-4a20-92f0-be3d99c4e37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:40.719865 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-361b4faa-af42-4a20-92f0-be3d99c4e37e None None] MFA Rules not processed for user `19af2d1084c14c3ea2df8369855aaed0`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:04:40.721222 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-361b4faa-af42-4a20-92f0-be3d99c4e37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:40.741523 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:04:40.798384 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1450/2905] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:04:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:04:40.802960 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc332767-cd1c-4769-8758-341e38838c5f service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:40.814492 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc332767-cd1c-4769-8758-341e38838c5f service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:40.819211 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c130ce-549f-4ee7-bf4c-bc45a1a48eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:40.819841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c130ce-549f-4ee7-bf4c-bc45a1a48eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:40.820147 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c130ce-549f-4ee7-bf4c-bc45a1a48eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:40.820422 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c130ce-549f-4ee7-bf4c-bc45a1a48eb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:40.821060 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c130ce-549f-4ee7-bf4c-bc45a1a48eb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:40.822975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c130ce-549f-4ee7-bf4c-bc45a1a48eb6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:40.852290 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1456/2906] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Sep 7 08:04:40 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 08:04:40.856851 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b4faa-af42-4a20-92f0-be3d99c4e37e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:40.867790 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b4faa-af42-4a20-92f0-be3d99c4e37e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:40.935419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f121831-db2e-43b8-a2b6-bd6545b6cc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:40.936654 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f121831-db2e-43b8-a2b6-bd6545b6cc8c admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:40.937173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f121831-db2e-43b8-a2b6-bd6545b6cc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:40.937670 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f121831-db2e-43b8-a2b6-bd6545b6cc8c admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:40.938658 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f121831-db2e-43b8-a2b6-bd6545b6cc8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:40.940671 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f121831-db2e-43b8-a2b6-bd6545b6cc8c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:40.968164 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1451/2907] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 7 08:04:40 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 08:04:40.972049 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c130ce-549f-4ee7-bf4c-bc45a1a48eb6 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:40.983373 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c130ce-549f-4ee7-bf4c-bc45a1a48eb6 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:40.988521 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcf3b2-94e1-49a4-b283-e7c99a770393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:40.989136 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcf3b2-94e1-49a4-b283-e7c99a770393 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:40.989447 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcf3b2-94e1-49a4-b283-e7c99a770393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:40.990278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcf3b2-94e1-49a4-b283-e7c99a770393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:40.990598 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdcf3b2-94e1-49a4-b283-e7c99a770393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:40.993086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdcf3b2-94e1-49a4-b283-e7c99a770393 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:41.016361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1457/2908] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 08:04:41.019666 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f121831-db2e-43b8-a2b6-bd6545b6cc8c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:41.030132 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f121831-db2e-43b8-a2b6-bd6545b6cc8c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:41.035123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95029bd1-18b3-42c2-a5ad-43275a1279be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:41.035782 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95029bd1-18b3-42c2-a5ad-43275a1279be admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:41.036062 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95029bd1-18b3-42c2-a5ad-43275a1279be admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:41.036362 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95029bd1-18b3-42c2-a5ad-43275a1279be admin admin] PATH_INFO: `/v3/projects/ead313b17824440eb3dd7f93583ca1b3/users/95b4f149c4024aa1b08a6ae6569ec298/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:41.051943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95029bd1-18b3-42c2-a5ad-43275a1279be admin admin] RBAC: Authorizing `identity:create_grant(project_id=ead313b17824440eb3dd7f93583ca1b3, user_id=95b4f149c4024aa1b08a6ae6569ec298, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:41.054632 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95029bd1-18b3-42c2-a5ad-43275a1279be admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:41.071260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1452/2909] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:04:41 2022] PUT /identity/v3/projects/ead313b17824440eb3dd7f93583ca1b3/users/95b4f149c4024aa1b08a6ae6569ec298/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:04:41.075787 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcf3b2-94e1-49a4-b283-e7c99a770393 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:41.086553 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcf3b2-94e1-49a4-b283-e7c99a770393 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:41.152739 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa3ec2-9a8e-425c-a276-c4f907fd3309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:41.153477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa3ec2-9a8e-425c-a276-c4f907fd3309 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:41.153762 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa3ec2-9a8e-425c-a276-c4f907fd3309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:41.154013 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa3ec2-9a8e-425c-a276-c4f907fd3309 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:41.155037 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa3ec2-9a8e-425c-a276-c4f907fd3309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:41.157228 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa3ec2-9a8e-425c-a276-c4f907fd3309 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:41.169651 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1458/2910] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Sep 7 08:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 08:04:41.176696 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95029bd1-18b3-42c2-a5ad-43275a1279be admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:41.191927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95029bd1-18b3-42c2-a5ad-43275a1279be admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:41.196734 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794a9c3-0e94-443c-96ec-8adc9b9ae3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:41.197391 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794a9c3-0e94-443c-96ec-8adc9b9ae3c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:41.197669 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794a9c3-0e94-443c-96ec-8adc9b9ae3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:41.197943 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794a9c3-0e94-443c-96ec-8adc9b9ae3c4 admin admin] PATH_INFO: `/v3/projects/ead313b17824440eb3dd7f93583ca1b3/users/95b4f149c4024aa1b08a6ae6569ec298/roles/202f2746fd624f55ad25e548a4db9ed8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:41.200347 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794a9c3-0e94-443c-96ec-8adc9b9ae3c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ead313b17824440eb3dd7f93583ca1b3, user_id=95b4f149c4024aa1b08a6ae6569ec298, role_id=202f2746fd624f55ad25e548a4db9ed8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:41.202990 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794a9c3-0e94-443c-96ec-8adc9b9ae3c4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:41.214370 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1453/2911] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Sep 7 08:04:41 2022] PUT /identity/v3/projects/ead313b17824440eb3dd7f93583ca1b3/users/95b4f149c4024aa1b08a6ae6569ec298/roles/202f2746fd624f55ad25e548a4db9ed8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:04:46.378234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5aa4bc-8089-43cf-853f-2f30e9e134ee None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:46.378728 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5aa4bc-8089-43cf-853f-2f30e9e134ee None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:46.378841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5aa4bc-8089-43cf-853f-2f30e9e134ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:46.421268 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7f5aa4bc-8089-43cf-853f-2f30e9e134ee None None] MFA Rules not processed for user `95b4f149c4024aa1b08a6ae6569ec298`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:04:46.422537 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7f5aa4bc-8089-43cf-853f-2f30e9e134ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:46.528354 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1459/2912] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:04:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:04:46.533444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794a9c3-0e94-443c-96ec-8adc9b9ae3c4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:04:46.535725 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f794a9c3-0e94-443c-96ec-8adc9b9ae3c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:46.634308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:04:46.668565 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794a9c3-0e94-443c-96ec-8adc9b9ae3c4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:04:46.676639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a38ba-bcf8-4e90-8fbe-2550926b53d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:04:46.677544 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a38ba-bcf8-4e90-8fbe-2550926b53d3 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:46.678036 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a38ba-bcf8-4e90-8fbe-2550926b53d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:46.678499 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a38ba-bcf8-4e90-8fbe-2550926b53d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:46.741805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a38ba-bcf8-4e90-8fbe-2550926b53d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:04:46.743397 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a38ba-bcf8-4e90-8fbe-2550926b53d3 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:04:46.753195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1454/2913] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:04:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:04:55.477417 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e213a6e-0218-45d9-ae61-c4c04c5f3b09 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:04:55.477814 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e213a6e-0218-45d9-ae61-c4c04c5f3b09 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:04:55.477955 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e213a6e-0218-45d9-ae61-c4c04c5f3b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:04:55.497617 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1e213a6e-0218-45d9-ae61-c4c04c5f3b09 None None] MFA Rules not processed for user `aef73a49de304706a59335eff37debf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:04:55.498813 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1e213a6e-0218-45d9-ae61-c4c04c5f3b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:04:55.505018 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:04:55.558551 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1460/2914] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:04:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:04:56.748937 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=33 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:05:05.551690 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=20 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:05:07.364025 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0ecc5-08af-4c98-8b10-b0e07e96f737 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:05:07.364524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0ecc5-08af-4c98-8b10-b0e07e96f737 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:05:07.364623 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0ecc5-08af-4c98-8b10-b0e07e96f737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:05:07.383953 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-99d0ecc5-08af-4c98-8b10-b0e07e96f737 None None] MFA Rules not processed for user `19af2d1084c14c3ea2df8369855aaed0`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 08:05:07.385227 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-99d0ecc5-08af-4c98-8b10-b0e07e96f737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:05:07.393127 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1455/2915] 10.222.0.26 () {58 vars in 958 bytes} [Wed Sep 7 08:05:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 08:05:15.117284 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e213a6e-0218-45d9-ae61-c4c04c5f3b09 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:05:15.119140 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1e213a6e-0218-45d9-ae61-c4c04c5f3b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:05:15.182455 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e213a6e-0218-45d9-ae61-c4c04c5f3b09 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:05:15.186267 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc124b71-f861-4a9c-9796-9c242c403ddb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:05:15.186869 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc124b71-f861-4a9c-9796-9c242c403ddb service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:05:15.187143 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc124b71-f861-4a9c-9796-9c242c403ddb service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:05:15.187411 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc124b71-f861-4a9c-9796-9c242c403ddb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:05:15.189157 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fc124b71-f861-4a9c-9796-9c242c403ddb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:05:15.203021 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc124b71-f861-4a9c-9796-9c242c403ddb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:05:15.204562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc124b71-f861-4a9c-9796-9c242c403ddb service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:05:15.216407 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1461/2916] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:05:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:05:17.378454 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:05:23.451254 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0ecc5-08af-4c98-8b10-b0e07e96f737 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:05:23.461141 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0ecc5-08af-4c98-8b10-b0e07e96f737 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:05:23.465290 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aff762-ade6-492e-b93c-13ebb73e1e1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:05:23.465881 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aff762-ade6-492e-b93c-13ebb73e1e1b service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:05:23.466158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aff762-ade6-492e-b93c-13ebb73e1e1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:05:23.466427 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aff762-ade6-492e-b93c-13ebb73e1e1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:05:23.468260 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-64aff762-ade6-492e-b93c-13ebb73e1e1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:05:23.482042 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aff762-ade6-492e-b93c-13ebb73e1e1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:05:23.483472 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aff762-ade6-492e-b93c-13ebb73e1e1b service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:05:23.495675 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1456/2917] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:05:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:05:25.210641 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=26 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:05:33.489711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:06:13.368581 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc124b71-f861-4a9c-9796-9c242c403ddb service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:06:13.377294 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc124b71-f861-4a9c-9796-9c242c403ddb service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:06:13.380981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf19f53-464f-4ba4-9d1c-82e8277c16c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:06:13.381571 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf19f53-464f-4ba4-9d1c-82e8277c16c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:06:13.381843 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf19f53-464f-4ba4-9d1c-82e8277c16c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:06:13.382109 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf19f53-464f-4ba4-9d1c-82e8277c16c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:06:13.383943 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8cf19f53-464f-4ba4-9d1c-82e8277c16c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:06:13.394500 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf19f53-464f-4ba4-9d1c-82e8277c16c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:06:13.395916 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf19f53-464f-4ba4-9d1c-82e8277c16c4 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:06:13.403697 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1462/2918] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:06:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:06:17.376854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aff762-ade6-492e-b93c-13ebb73e1e1b service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:06:17.389513 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aff762-ade6-492e-b93c-13ebb73e1e1b service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:06:17.395552 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07aa21-352a-4a08-8ec4-06ad61a370d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:06:17.396214 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07aa21-352a-4a08-8ec4-06ad61a370d2 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:06:17.396494 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07aa21-352a-4a08-8ec4-06ad61a370d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:06:17.397194 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07aa21-352a-4a08-8ec4-06ad61a370d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:06:17.398481 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ff07aa21-352a-4a08-8ec4-06ad61a370d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:06:17.411486 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07aa21-352a-4a08-8ec4-06ad61a370d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:06:17.413089 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07aa21-352a-4a08-8ec4-06ad61a370d2 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:06:17.424989 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1457/2919] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:06:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:06:20.389968 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf19f53-464f-4ba4-9d1c-82e8277c16c4 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:06:20.399011 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf19f53-464f-4ba4-9d1c-82e8277c16c4 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:06:20.402516 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b86500-be32-4a49-bed5-54e0c9986567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:06:20.403093 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b86500-be32-4a49-bed5-54e0c9986567 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:06:20.403372 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b86500-be32-4a49-bed5-54e0c9986567 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:06:20.403637 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b86500-be32-4a49-bed5-54e0c9986567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:06:20.412933 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b86500-be32-4a49-bed5-54e0c9986567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:06:20.414362 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b86500-be32-4a49-bed5-54e0c9986567 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:06:20.424321 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1463/2920] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:06:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:06:22.137993 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07aa21-352a-4a08-8ec4-06ad61a370d2 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:06:22.139421 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ff07aa21-352a-4a08-8ec4-06ad61a370d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:06:22.153353 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07aa21-352a-4a08-8ec4-06ad61a370d2 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:06:22.160019 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3636eb-084f-40a0-95a4-0f48daaa17a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:06:22.160646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3636eb-084f-40a0-95a4-0f48daaa17a0 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:06:22.160923 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3636eb-084f-40a0-95a4-0f48daaa17a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:06:22.161195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3636eb-084f-40a0-95a4-0f48daaa17a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:06:22.162916 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1a3636eb-084f-40a0-95a4-0f48daaa17a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:06:22.175996 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3636eb-084f-40a0-95a4-0f48daaa17a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:06:22.177572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3636eb-084f-40a0-95a4-0f48daaa17a0 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:06:22.188794 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1458/2921] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:06:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:06:30.419562 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=12 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:06:32.183646 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=12 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:08:22.951010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b86500-be32-4a49-bed5-54e0c9986567 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:08:22.966231 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b86500-be32-4a49-bed5-54e0c9986567 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:08:22.982322 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a9e2b-f510-4f4a-ab8e-31a6f91622ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:08:22.982957 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a9e2b-f510-4f4a-ab8e-31a6f91622ce service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:08:22.983385 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a9e2b-f510-4f4a-ab8e-31a6f91622ce service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:08:22.983531 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a9e2b-f510-4f4a-ab8e-31a6f91622ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:08:22.986141 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f47a9e2b-f510-4f4a-ab8e-31a6f91622ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:08:23.001496 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a9e2b-f510-4f4a-ab8e-31a6f91622ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:08:23.003481 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a9e2b-f510-4f4a-ab8e-31a6f91622ce service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:08:23.015998 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1464/2922] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:08:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:08:33.010451 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=6 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:08:45.793813 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3636eb-084f-40a0-95a4-0f48daaa17a0 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:08:45.795301 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1a3636eb-084f-40a0-95a4-0f48daaa17a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:08:45.853820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3636eb-084f-40a0-95a4-0f48daaa17a0 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:08:45.857443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b609e5-fd82-412c-9cf4-0906a7970758 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:08:45.858038 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b609e5-fd82-412c-9cf4-0906a7970758 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:08:45.858316 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b609e5-fd82-412c-9cf4-0906a7970758 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:08:45.858587 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b609e5-fd82-412c-9cf4-0906a7970758 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:08:45.860338 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e9b609e5-fd82-412c-9cf4-0906a7970758 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:08:45.871835 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b609e5-fd82-412c-9cf4-0906a7970758 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:08:45.873266 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b609e5-fd82-412c-9cf4-0906a7970758 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:08:45.882212 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1459/2923] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:08:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:08:55.878101 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=26 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:09:32.551427 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a9e2b-f510-4f4a-ab8e-31a6f91622ce service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:09:32.552816 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f47a9e2b-f510-4f4a-ab8e-31a6f91622ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:09:32.620954 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a9e2b-f510-4f4a-ab8e-31a6f91622ce service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:09:32.626123 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d36f66-59d4-4c0b-8f02-19e099429c41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:09:32.626734 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d36f66-59d4-4c0b-8f02-19e099429c41 service placement] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:09:32.627010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d36f66-59d4-4c0b-8f02-19e099429c41 service placement] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:09:32.627278 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d36f66-59d4-4c0b-8f02-19e099429c41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:09:32.628959 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-60d36f66-59d4-4c0b-8f02-19e099429c41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:09:32.640430 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d36f66-59d4-4c0b-8f02-19e099429c41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:09:32.642010 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d36f66-59d4-4c0b-8f02-19e099429c41 service placement] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:09:32.653212 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1465/2924] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:09:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:09:42.648088 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=32 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:09:46.348718 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b609e5-fd82-412c-9cf4-0906a7970758 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:09:46.361197 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b609e5-fd82-412c-9cf4-0906a7970758 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:09:46.366616 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa1a28-e7bb-4bf5-a6ed-2ada32b17eab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:09:46.367151 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa1a28-e7bb-4bf5-a6ed-2ada32b17eab service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:09:46.367429 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa1a28-e7bb-4bf5-a6ed-2ada32b17eab service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:09:46.367698 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa1a28-e7bb-4bf5-a6ed-2ada32b17eab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:09:46.375346 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa1a28-e7bb-4bf5-a6ed-2ada32b17eab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:09:46.376942 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa1a28-e7bb-4bf5-a6ed-2ada32b17eab service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:09:46.386064 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1460/2925] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:09:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:09:56.381874 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:10:16.260323 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d36f66-59d4-4c0b-8f02-19e099429c41 service placement] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:10:16.269756 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d36f66-59d4-4c0b-8f02-19e099429c41 service placement] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:10:16.273260 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581ed18-780b-4d76-ad93-7becd41005e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:10:16.273841 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581ed18-780b-4d76-ad93-7becd41005e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:10:16.274113 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581ed18-780b-4d76-ad93-7becd41005e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:10:16.274441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581ed18-780b-4d76-ad93-7becd41005e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:10:16.284378 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9581ed18-780b-4d76-ad93-7becd41005e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:10:16.285828 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9581ed18-780b-4d76-ad93-7becd41005e5 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:10:16.297051 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1466/2926] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:10:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:10:24.025733 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa1a28-e7bb-4bf5-a6ed-2ada32b17eab service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:10:24.035011 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa1a28-e7bb-4bf5-a6ed-2ada32b17eab service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:10:24.038559 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a389a0-4092-466b-a412-cef5abcf6f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:10:24.039146 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a389a0-4092-466b-a412-cef5abcf6f66 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:10:24.039424 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a389a0-4092-466b-a412-cef5abcf6f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:10:24.039694 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a389a0-4092-466b-a412-cef5abcf6f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:10:24.049902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a389a0-4092-466b-a412-cef5abcf6f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:10:24.051299 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a389a0-4092-466b-a412-cef5abcf6f66 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:10:24.062310 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1461/2927] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:10:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:10:26.291634 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=6 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:10:34.057227 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:11:31.191736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581ed18-780b-4d76-ad93-7becd41005e5 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:11:31.200495 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581ed18-780b-4d76-ad93-7becd41005e5 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:11:31.204025 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4401d-ba91-4952-bf9d-83438d0cf2dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:11:31.204630 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4401d-ba91-4952-bf9d-83438d0cf2dd service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:11:31.204929 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4401d-ba91-4952-bf9d-83438d0cf2dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:11:31.205197 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4401d-ba91-4952-bf9d-83438d0cf2dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:11:31.212414 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4401d-ba91-4952-bf9d-83438d0cf2dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:11:31.213836 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4401d-ba91-4952-bf9d-83438d0cf2dd service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:11:31.222204 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1467/2928] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:11:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:11:41.218121 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=6 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:14:23.318160 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a389a0-4092-466b-a412-cef5abcf6f66 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:14:23.355642 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a389a0-4092-466b-a412-cef5abcf6f66 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:14:23.359200 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf420e6f-080c-47b4-89e6-631d483cd953 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:14:23.359778 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf420e6f-080c-47b4-89e6-631d483cd953 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:14:23.360048 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf420e6f-080c-47b4-89e6-631d483cd953 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:14:23.360342 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf420e6f-080c-47b4-89e6-631d483cd953 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:14:23.381578 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf420e6f-080c-47b4-89e6-631d483cd953 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:14:23.383096 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf420e6f-080c-47b4-89e6-631d483cd953 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:14:23.410333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1462/2929] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:14:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:14:33.407822 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=31 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:14:39.506649 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4401d-ba91-4952-bf9d-83438d0cf2dd service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:14:39.545602 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4401d-ba91-4952-bf9d-83438d0cf2dd service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:14:39.550607 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c047d3-ef55-45e0-80ed-18a0e35d4090 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:14:39.551195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c047d3-ef55-45e0-80ed-18a0e35d4090 service placement] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:14:39.551473 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c047d3-ef55-45e0-80ed-18a0e35d4090 service placement] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:14:39.551743 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c047d3-ef55-45e0-80ed-18a0e35d4090 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:14:39.561198 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c047d3-ef55-45e0-80ed-18a0e35d4090 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:14:39.562753 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c047d3-ef55-45e0-80ed-18a0e35d4090 service placement] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:14:39.572690 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1468/2930] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:14:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:14:47.413707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf420e6f-080c-47b4-89e6-631d483cd953 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:14:47.498617 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf420e6f-080c-47b4-89e6-631d483cd953 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:14:47.504149 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fef10-8a16-4a0d-a781-e75a8e460342 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:14:47.504738 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fef10-8a16-4a0d-a781-e75a8e460342 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:14:47.505018 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fef10-8a16-4a0d-a781-e75a8e460342 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:14:47.505293 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fef10-8a16-4a0d-a781-e75a8e460342 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:14:47.507123 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e32fef10-8a16-4a0d-a781-e75a8e460342 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:14:47.594877 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32fef10-8a16-4a0d-a781-e75a8e460342 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:14:47.596471 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32fef10-8a16-4a0d-a781-e75a8e460342 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:14:47.614664 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1463/2931] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:14:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:14:49.568368 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=18 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:14:57.612104 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=80 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:15:16.362585 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c047d3-ef55-45e0-80ed-18a0e35d4090 service placement] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:15:16.364181 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-91c047d3-ef55-45e0-80ed-18a0e35d4090 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:15:16.441351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c047d3-ef55-45e0-80ed-18a0e35d4090 service placement] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:15:16.444937 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032d3ed-7e0f-4d14-9133-fd6c8701860e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:15:16.445534 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032d3ed-7e0f-4d14-9133-fd6c8701860e service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:15:16.445812 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032d3ed-7e0f-4d14-9133-fd6c8701860e service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:15:16.446086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032d3ed-7e0f-4d14-9133-fd6c8701860e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:15:16.448499 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3032d3ed-7e0f-4d14-9133-fd6c8701860e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:15:16.473815 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032d3ed-7e0f-4d14-9133-fd6c8701860e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:15:16.475258 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032d3ed-7e0f-4d14-9133-fd6c8701860e service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:15:16.486466 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1469/2932] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:15:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:15:24.216746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fef10-8a16-4a0d-a781-e75a8e460342 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:15:24.218308 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e32fef10-8a16-4a0d-a781-e75a8e460342 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:15:24.229862 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fef10-8a16-4a0d-a781-e75a8e460342 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:15:24.233409 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312fb2e-2081-44f0-9b24-c59c461a7867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:15:24.233997 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312fb2e-2081-44f0-9b24-c59c461a7867 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:15:24.234275 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312fb2e-2081-44f0-9b24-c59c461a7867 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:15:24.234546 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312fb2e-2081-44f0-9b24-c59c461a7867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:15:24.236480 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3312fb2e-2081-44f0-9b24-c59c461a7867 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:15:24.250128 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312fb2e-2081-44f0-9b24-c59c461a7867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:15:24.251532 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312fb2e-2081-44f0-9b24-c59c461a7867 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:15:24.262998 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1464/2933] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:15:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:15:26.481441 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=40 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:15:34.257433 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:16:34.163789 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032d3ed-7e0f-4d14-9133-fd6c8701860e service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:16:34.172877 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032d3ed-7e0f-4d14-9133-fd6c8701860e service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:16:34.176412 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967255cc-f74e-482a-9e32-c7577d695c29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:16:34.176997 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967255cc-f74e-482a-9e32-c7577d695c29 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:16:34.177270 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967255cc-f74e-482a-9e32-c7577d695c29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:16:34.177535 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967255cc-f74e-482a-9e32-c7577d695c29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:16:34.179349 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-967255cc-f74e-482a-9e32-c7577d695c29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:16:34.189981 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967255cc-f74e-482a-9e32-c7577d695c29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:16:34.191404 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967255cc-f74e-482a-9e32-c7577d695c29 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:16:34.199361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1470/2934] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:16:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:16:44.195755 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=6 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:19:24.352830 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312fb2e-2081-44f0-9b24-c59c461a7867 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:19:24.354463 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3312fb2e-2081-44f0-9b24-c59c461a7867 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:19:24.369102 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312fb2e-2081-44f0-9b24-c59c461a7867 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:19:24.374514 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70725d5-9e11-4477-a95e-f2bc4d3341b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:19:24.375105 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70725d5-9e11-4477-a95e-f2bc4d3341b3 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:19:24.375381 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70725d5-9e11-4477-a95e-f2bc4d3341b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:19:24.375654 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70725d5-9e11-4477-a95e-f2bc4d3341b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:19:24.377442 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b70725d5-9e11-4477-a95e-f2bc4d3341b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:19:24.447746 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70725d5-9e11-4477-a95e-f2bc4d3341b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:19:24.449339 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70725d5-9e11-4477-a95e-f2bc4d3341b3 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:19:24.458195 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1465/2935] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:19:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:19:34.454318 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=33 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:19:42.479068 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967255cc-f74e-482a-9e32-c7577d695c29 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:19:42.480486 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-967255cc-f74e-482a-9e32-c7577d695c29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:19:42.560574 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967255cc-f74e-482a-9e32-c7577d695c29 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:19:42.565628 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e562be-2290-43d9-a619-2411a75f08d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:19:42.566222 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e562be-2290-43d9-a619-2411a75f08d1 service placement] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:19:42.566498 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e562be-2290-43d9-a619-2411a75f08d1 service placement] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:19:42.566769 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e562be-2290-43d9-a619-2411a75f08d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:19:42.568616 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a1e562be-2290-43d9-a619-2411a75f08d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:19:42.579861 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e562be-2290-43d9-a619-2411a75f08d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:19:42.581457 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e562be-2290-43d9-a619-2411a75f08d1 service placement] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:19:42.591141 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1471/2936] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:19:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:19:47.912289 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70725d5-9e11-4477-a95e-f2bc4d3341b3 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:19:47.913703 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b70725d5-9e11-4477-a95e-f2bc4d3341b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:19:47.981466 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70725d5-9e11-4477-a95e-f2bc4d3341b3 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:19:47.985683 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafc9d1-24c6-4111-9fd0-64f58b792009 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:19:47.986418 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafc9d1-24c6-4111-9fd0-64f58b792009 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:19:47.986695 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafc9d1-24c6-4111-9fd0-64f58b792009 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:19:47.986958 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafc9d1-24c6-4111-9fd0-64f58b792009 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:19:47.995382 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaafc9d1-24c6-4111-9fd0-64f58b792009 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:19:47.996839 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaafc9d1-24c6-4111-9fd0-64f58b792009 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:19:48.006447 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1466/2937] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:19:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:19:52.587028 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=37 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:19:58.002332 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=31 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:20:17.348949 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e562be-2290-43d9-a619-2411a75f08d1 service placement] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:20:17.358202 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e562be-2290-43d9-a619-2411a75f08d1 service placement] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:20:17.361801 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f108e-cb8e-4494-a211-aa5665ad869a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:20:17.362387 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f108e-cb8e-4494-a211-aa5665ad869a service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:20:17.362660 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f108e-cb8e-4494-a211-aa5665ad869a service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:20:17.362919 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f108e-cb8e-4494-a211-aa5665ad869a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:20:17.372824 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f108e-cb8e-4494-a211-aa5665ad869a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:20:17.374227 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f108e-cb8e-4494-a211-aa5665ad869a service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:20:17.385111 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1472/2938] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:20:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:20:23.245561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafc9d1-24c6-4111-9fd0-64f58b792009 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:20:23.255026 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafc9d1-24c6-4111-9fd0-64f58b792009 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:20:23.258586 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb581247-d08e-48ac-9499-6926164adeb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:20:23.259173 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb581247-d08e-48ac-9499-6926164adeb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:20:23.259453 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb581247-d08e-48ac-9499-6926164adeb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:20:23.259726 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb581247-d08e-48ac-9499-6926164adeb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:20:23.270470 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb581247-d08e-48ac-9499-6926164adeb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:20:23.271880 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb581247-d08e-48ac-9499-6926164adeb2 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:20:23.282685 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1467/2939] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:20:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:20:27.380244 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=6 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:20:33.277902 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:21:18.039953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f108e-cb8e-4494-a211-aa5665ad869a service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:21:18.041835 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-781f108e-cb8e-4494-a211-aa5665ad869a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:21:18.057335 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f108e-cb8e-4494-a211-aa5665ad869a service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:21:18.063592 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41fd41-dfeb-46fd-8069-9fe2b4154875 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81d5689e9a344bafb74ca423ffd2b42a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:21:18.064235 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41fd41-dfeb-46fd-8069-9fe2b4154875 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:21:18.064579 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41fd41-dfeb-46fd-8069-9fe2b4154875 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:21:18.064855 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41fd41-dfeb-46fd-8069-9fe2b4154875 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:21:18.067938 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3c41fd41-dfeb-46fd-8069-9fe2b4154875 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:21:18.081012 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41fd41-dfeb-46fd-8069-9fe2b4154875 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:21:18.082597 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41fd41-dfeb-46fd-8069-9fe2b4154875 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:21:18.104477 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1473/2940] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:21:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 08:21:28.094276 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=6 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:24:23.870210 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb581247-d08e-48ac-9499-6926164adeb2 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:24:23.897048 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb581247-d08e-48ac-9499-6926164adeb2 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:24:23.900594 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a2b6a-05fc-4c69-98ae-ce411540c366 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:24:23.901176 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a2b6a-05fc-4c69-98ae-ce411540c366 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:24:23.901450 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a2b6a-05fc-4c69-98ae-ce411540c366 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:24:23.901719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a2b6a-05fc-4c69-98ae-ce411540c366 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:24:23.923354 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a2b6a-05fc-4c69-98ae-ce411540c366 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:24:23.925481 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a2b6a-05fc-4c69-98ae-ce411540c366 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:24:23.951854 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1468/2941] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:24:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:24:26.168230 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41fd41-dfeb-46fd-8069-9fe2b4154875 service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:24:26.178736 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41fd41-dfeb-46fd-8069-9fe2b4154875 service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:24:26.182879 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59236dfe-a627-4b6c-9ef9-5583ce60129c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:24:26.183483 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59236dfe-a627-4b6c-9ef9-5583ce60129c service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:24:26.183765 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59236dfe-a627-4b6c-9ef9-5583ce60129c service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:24:26.184038 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59236dfe-a627-4b6c-9ef9-5583ce60129c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:24:26.186283 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-59236dfe-a627-4b6c-9ef9-5583ce60129c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:24:26.273434 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59236dfe-a627-4b6c-9ef9-5583ce60129c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:24:26.274940 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59236dfe-a627-4b6c-9ef9-5583ce60129c service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:24:26.296975 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1474/2942] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:24:29.398158 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a2b6a-05fc-4c69-98ae-ce411540c366 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:24:29.399753 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6a7a2b6a-05fc-4c69-98ae-ce411540c366 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:24:29.421745 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a2b6a-05fc-4c69-98ae-ce411540c366 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:24:29.429907 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60b0f9-4e6f-4b7e-a5b8-98382fa19629 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:24:29.431004 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60b0f9-4e6f-4b7e-a5b8-98382fa19629 service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:24:29.431356 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60b0f9-4e6f-4b7e-a5b8-98382fa19629 service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:24:29.431631 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60b0f9-4e6f-4b7e-a5b8-98382fa19629 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:24:29.433895 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1d60b0f9-4e6f-4b7e-a5b8-98382fa19629 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:24:29.448121 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d60b0f9-4e6f-4b7e-a5b8-98382fa19629 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:24:29.450174 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d60b0f9-4e6f-4b7e-a5b8-98382fa19629 service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:24:29.470575 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1469/2943] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:24:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:24:32.318784 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59236dfe-a627-4b6c-9ef9-5583ce60129c service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:24:32.330015 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59236dfe-a627-4b6c-9ef9-5583ce60129c service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:24:32.336086 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6088d-249c-42fd-9674-98a5049978d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:24:32.336824 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6088d-249c-42fd-9674-98a5049978d6 service placement] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:24:32.337099 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6088d-249c-42fd-9674-98a5049978d6 service placement] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:24:32.337367 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6088d-249c-42fd-9674-98a5049978d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:24:32.339064 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8bf6088d-249c-42fd-9674-98a5049978d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:24:32.351749 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6088d-249c-42fd-9674-98a5049978d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:24:32.353382 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6088d-249c-42fd-9674-98a5049978d6 service placement] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:24:32.363995 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1475/2944] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:24:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:24:32.397110 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60b0f9-4e6f-4b7e-a5b8-98382fa19629 service placement] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:24:32.399333 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1d60b0f9-4e6f-4b7e-a5b8-98382fa19629 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:24:32.422366 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60b0f9-4e6f-4b7e-a5b8-98382fa19629 service placement] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:24:32.427496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b8638-df7f-43e1-a546-4862dfab471c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:24:32.428154 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b8638-df7f-43e1-a546-4862dfab471c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:24:32.428436 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b8638-df7f-43e1-a546-4862dfab471c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:24:32.428705 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b8638-df7f-43e1-a546-4862dfab471c admin admin] PATH_INFO: `/v3/users/19af2d1084c14c3ea2df8369855aaed0` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:24:32.429826 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b8638-df7f-43e1-a546-4862dfab471c admin admin] RBAC: Authorizing `identity:delete_user(user_id=19af2d1084c14c3ea2df8369855aaed0)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:24:32.432211 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b8638-df7f-43e1-a546-4862dfab471c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:24:32.484586 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8b0b8638-df7f-43e1-a546-4862dfab471c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19af2d1084c14c3ea2df8369855aaed0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:32.493281 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8b0b8638-df7f-43e1-a546-4862dfab471c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19af2d1084c14c3ea2df8369855aaed0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:32.500639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8b0b8638-df7f-43e1-a546-4862dfab471c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19af2d1084c14c3ea2df8369855aaed0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:32.501326 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8b0b8638-df7f-43e1-a546-4862dfab471c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19af2d1084c14c3ea2df8369855aaed0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:32.509051 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8b0b8638-df7f-43e1-a546-4862dfab471c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19af2d1084c14c3ea2df8369855aaed0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:32.514736 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1470/2945] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:24:32 2022] DELETE /identity/v3/users/19af2d1084c14c3ea2df8369855aaed0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:24:32.691095 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6088d-249c-42fd-9674-98a5049978d6 service placement] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:24:32.701663 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6088d-249c-42fd-9674-98a5049978d6 service placement] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:24:32.758410 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0537b-7714-409f-92c4-437d3a27153f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:24:32.759033 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0537b-7714-409f-92c4-437d3a27153f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:24:32.759361 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0537b-7714-409f-92c4-437d3a27153f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:24:32.759580 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0537b-7714-409f-92c4-437d3a27153f admin admin] PATH_INFO: `/v3/projects/e2744f91a6284e0688af92a926c199d1` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:24:32.760678 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0537b-7714-409f-92c4-437d3a27153f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2744f91a6284e0688af92a926c199d1)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:24:32.762578 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0537b-7714-409f-92c4-437d3a27153f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:24:32.808173 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a9b0537b-7714-409f-92c4-437d3a27153f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2744f91a6284e0688af92a926c199d1', 'request_id': 'req-a9b0537b-7714-409f-92c4-437d3a27153f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:32.811874 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:24:32.814302 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a9b0537b-7714-409f-92c4-437d3a27153f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2744f91a6284e0688af92a926c199d1', 'request_id': 'req-a9b0537b-7714-409f-92c4-437d3a27153f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:32.822133 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a9b0537b-7714-409f-92c4-437d3a27153f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2744f91a6284e0688af92a926c199d1', 'request_id': 'req-a9b0537b-7714-409f-92c4-437d3a27153f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:32.836255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a9b0537b-7714-409f-92c4-437d3a27153f admin admin] The token cache is being invalidate because project e2744f91a6284e0688af92a926c199d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:24:32.836485 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a9b0537b-7714-409f-92c4-437d3a27153f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:32.837103 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a9b0537b-7714-409f-92c4-437d3a27153f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:32.838676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1476/2946] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:24:32 2022] DELETE /identity/v3/projects/e2744f91a6284e0688af92a926c199d1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 08:24:32.842648 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b8638-df7f-43e1-a546-4862dfab471c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:24:32.844188 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8b0b8638-df7f-43e1-a546-4862dfab471c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:24:32.857633 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b8638-df7f-43e1-a546-4862dfab471c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:24:32.915748 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cd96c-07a6-4fde-9abd-b8e42a8fa54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:24:32.916400 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cd96c-07a6-4fde-9abd-b8e42a8fa54d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:24:32.916678 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cd96c-07a6-4fde-9abd-b8e42a8fa54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:24:32.916946 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cd96c-07a6-4fde-9abd-b8e42a8fa54d admin admin] PATH_INFO: `/v3/users/95b4f149c4024aa1b08a6ae6569ec298` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:24:32.919701 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:24:32.933643 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cd96c-07a6-4fde-9abd-b8e42a8fa54d admin admin] RBAC: Authorizing `identity:delete_user(user_id=95b4f149c4024aa1b08a6ae6569ec298)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:24:32.935572 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cd96c-07a6-4fde-9abd-b8e42a8fa54d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:24:32.981900 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-424cd96c-07a6-4fde-9abd-b8e42a8fa54d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95b4f149c4024aa1b08a6ae6569ec298'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:32.994444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-424cd96c-07a6-4fde-9abd-b8e42a8fa54d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95b4f149c4024aa1b08a6ae6569ec298'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:33.001363 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-424cd96c-07a6-4fde-9abd-b8e42a8fa54d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95b4f149c4024aa1b08a6ae6569ec298'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:33.001968 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-424cd96c-07a6-4fde-9abd-b8e42a8fa54d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95b4f149c4024aa1b08a6ae6569ec298'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:33.009500 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-424cd96c-07a6-4fde-9abd-b8e42a8fa54d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95b4f149c4024aa1b08a6ae6569ec298'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:33.015482 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1471/2947] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:24:32 2022] DELETE /identity/v3/users/95b4f149c4024aa1b08a6ae6569ec298 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:24:33.187676 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0537b-7714-409f-92c4-437d3a27153f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:24:33.189084 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a9b0537b-7714-409f-92c4-437d3a27153f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:24:33.206561 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0537b-7714-409f-92c4-437d3a27153f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:24:33.264489 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1095a-326b-44ab-90e5-dee9685b9429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:24:33.265032 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1095a-326b-44ab-90e5-dee9685b9429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:24:33.265304 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1095a-326b-44ab-90e5-dee9685b9429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:24:33.265575 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1095a-326b-44ab-90e5-dee9685b9429 admin admin] PATH_INFO: `/v3/projects/ead313b17824440eb3dd7f93583ca1b3` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:24:33.275838 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1095a-326b-44ab-90e5-dee9685b9429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ead313b17824440eb3dd7f93583ca1b3)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:24:33.277831 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1095a-326b-44ab-90e5-dee9685b9429 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:24:33.324371 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5bb1095a-326b-44ab-90e5-dee9685b9429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ead313b17824440eb3dd7f93583ca1b3', 'request_id': 'req-5bb1095a-326b-44ab-90e5-dee9685b9429'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:33.330106 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5bb1095a-326b-44ab-90e5-dee9685b9429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ead313b17824440eb3dd7f93583ca1b3', 'request_id': 'req-5bb1095a-326b-44ab-90e5-dee9685b9429'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:33.337229 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5bb1095a-326b-44ab-90e5-dee9685b9429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ead313b17824440eb3dd7f93583ca1b3', 'request_id': 'req-5bb1095a-326b-44ab-90e5-dee9685b9429'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:33.349591 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5bb1095a-326b-44ab-90e5-dee9685b9429 admin admin] The token cache is being invalidate because project ead313b17824440eb3dd7f93583ca1b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:24:33.349866 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5bb1095a-326b-44ab-90e5-dee9685b9429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:33.350485 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5bb1095a-326b-44ab-90e5-dee9685b9429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:24:33.351953 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1477/2948] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:24:33 2022] DELETE /identity/v3/projects/ead313b17824440eb3dd7f93583ca1b3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:24:43.011892 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:24:43.346857 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=9,keystone:SELECT=60,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:24:47.503461 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cd96c-07a6-4fde-9abd-b8e42a8fa54d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:24:47.504927 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-424cd96c-07a6-4fde-9abd-b8e42a8fa54d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:24:47.599324 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cd96c-07a6-4fde-9abd-b8e42a8fa54d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:24:47.656580 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cfecf-7f67-4e0a-8d58-f82e7c5c71b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:24:47.657113 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cfecf-7f67-4e0a-8d58-f82e7c5c71b6 service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:24:47.657444 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cfecf-7f67-4e0a-8d58-f82e7c5c71b6 service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:24:47.657719 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cfecf-7f67-4e0a-8d58-f82e7c5c71b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:24:47.659547 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-709cfecf-7f67-4e0a-8d58-f82e7c5c71b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:24:47.679289 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cfecf-7f67-4e0a-8d58-f82e7c5c71b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:24:47.680870 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cfecf-7f67-4e0a-8d58-f82e7c5c71b6 service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:24:47.691300 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1472/2949] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:24:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:24:57.686659 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=74 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:29:23.607424 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1095a-326b-44ab-90e5-dee9685b9429 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:29:23.608829 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5bb1095a-326b-44ab-90e5-dee9685b9429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:29:23.668038 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1095a-326b-44ab-90e5-dee9685b9429 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:29:23.671509 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97448e49-725f-4337-a514-5034e02cc694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:29:23.672122 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97448e49-725f-4337-a514-5034e02cc694 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:29:23.672399 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97448e49-725f-4337-a514-5034e02cc694 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:29:23.672664 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97448e49-725f-4337-a514-5034e02cc694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:29:23.674309 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-97448e49-725f-4337-a514-5034e02cc694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:29:23.731039 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97448e49-725f-4337-a514-5034e02cc694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:29:23.732513 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97448e49-725f-4337-a514-5034e02cc694 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:29:23.741077 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1478/2950] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:29:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:29:33.737169 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=50 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:29:53.465751 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cfecf-7f67-4e0a-8d58-f82e7c5c71b6 service placement] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:29:53.467183 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-709cfecf-7f67-4e0a-8d58-f82e7c5c71b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:29:53.494786 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cfecf-7f67-4e0a-8d58-f82e7c5c71b6 service placement] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:29:53.499820 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43fade-0643-4627-8d77-6441b3c75598 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:29:53.500449 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43fade-0643-4627-8d77-6441b3c75598 service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:29:53.500729 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43fade-0643-4627-8d77-6441b3c75598 service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:29:53.500999 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43fade-0643-4627-8d77-6441b3c75598 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:29:53.509561 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43fade-0643-4627-8d77-6441b3c75598 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:29:53.511229 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43fade-0643-4627-8d77-6441b3c75598 service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:29:53.521071 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1473/2951] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:29:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:30:03.516805 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=11 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:34:24.040786 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97448e49-725f-4337-a514-5034e02cc694 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:34:24.067938 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97448e49-725f-4337-a514-5034e02cc694 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:34:24.071460 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f8a1b-76cd-4ba6-8c00-4adc37bf7e90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92699b512c064b05948afe597650fdb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:34:24.072046 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f8a1b-76cd-4ba6-8c00-4adc37bf7e90 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:34:24.072351 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f8a1b-76cd-4ba6-8c00-4adc37bf7e90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:34:24.072708 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f8a1b-76cd-4ba6-8c00-4adc37bf7e90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:34:24.094723 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f8a1b-76cd-4ba6-8c00-4adc37bf7e90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:34:24.096171 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f8a1b-76cd-4ba6-8c00-4adc37bf7e90 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:34:24.123404 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1479/2952] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:34:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:34:34.120538 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=26 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:35:00.488298 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43fade-0643-4627-8d77-6441b3c75598 service placement] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:35:00.489707 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bd43fade-0643-4627-8d77-6441b3c75598 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:35:00.602262 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43fade-0643-4627-8d77-6441b3c75598 service placement] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:35:00.609536 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfc823-4070-45a1-89c9-db407a60be5e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:35:00.610495 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfc823-4070-45a1-89c9-db407a60be5e service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:35:00.610961 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfc823-4070-45a1-89c9-db407a60be5e service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:35:00.611419 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfc823-4070-45a1-89c9-db407a60be5e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:35:00.614323 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5fcfc823-4070-45a1-89c9-db407a60be5e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:35:00.628496 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfc823-4070-45a1-89c9-db407a60be5e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:35:00.630045 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfc823-4070-45a1-89c9-db407a60be5e service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:35:00.641190 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1474/2953] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:35:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:35:10.635610 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=52 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:36:16.033240 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f8a1b-76cd-4ba6-8c00-4adc37bf7e90 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:36:16.035158 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1b5f8a1b-76cd-4ba6-8c00-4adc37bf7e90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:36:16.123773 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f8a1b-76cd-4ba6-8c00-4adc37bf7e90 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:36:16.127356 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc071dd6-e279-4096-ad7f-d2a316af186f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '330a38822d33439191773e9a8b4cbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:36:16.127948 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc071dd6-e279-4096-ad7f-d2a316af186f service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:36:16.128255 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc071dd6-e279-4096-ad7f-d2a316af186f service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:36:16.128523 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc071dd6-e279-4096-ad7f-d2a316af186f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:36:16.130190 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dc071dd6-e279-4096-ad7f-d2a316af186f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:36:16.213936 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc071dd6-e279-4096-ad7f-d2a316af186f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:36:16.215527 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc071dd6-e279-4096-ad7f-d2a316af186f service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:36:16.233741 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1480/2954] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:36:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:36:16.686608 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfc823-4070-45a1-89c9-db407a60be5e service placement] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:36:16.697828 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfc823-4070-45a1-89c9-db407a60be5e service placement] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:36:16.703385 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adca82-82b0-4b68-bf88-32aee8eca5d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:36:16.703982 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adca82-82b0-4b68-bf88-32aee8eca5d5 service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:36:16.704295 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adca82-82b0-4b68-bf88-32aee8eca5d5 service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:36:16.704566 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adca82-82b0-4b68-bf88-32aee8eca5d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:36:16.706298 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-93adca82-82b0-4b68-bf88-32aee8eca5d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:36:16.718711 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93adca82-82b0-4b68-bf88-32aee8eca5d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:36:16.720233 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93adca82-82b0-4b68-bf88-32aee8eca5d5 service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:36:16.734041 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1475/2955] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:36:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:36:16.738991 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc071dd6-e279-4096-ad7f-d2a316af186f service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:36:16.740560 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dc071dd6-e279-4096-ad7f-d2a316af186f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:36:16.753029 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc071dd6-e279-4096-ad7f-d2a316af186f service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:36:16.757999 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42f3eb-8cea-4314-a6c1-2ef3afd6bee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:36:16.758709 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42f3eb-8cea-4314-a6c1-2ef3afd6bee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:36:16.759076 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42f3eb-8cea-4314-a6c1-2ef3afd6bee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:36:16.759419 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42f3eb-8cea-4314-a6c1-2ef3afd6bee9 admin admin] PATH_INFO: `/v3/users/aef73a49de304706a59335eff37debf0` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:36:16.778616 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42f3eb-8cea-4314-a6c1-2ef3afd6bee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aef73a49de304706a59335eff37debf0)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:36:16.780614 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42f3eb-8cea-4314-a6c1-2ef3afd6bee9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:36:16.819085 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:36:16.826600 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3b42f3eb-8cea-4314-a6c1-2ef3afd6bee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aef73a49de304706a59335eff37debf0'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:36:16.827262 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3b42f3eb-8cea-4314-a6c1-2ef3afd6bee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aef73a49de304706a59335eff37debf0'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:36:16.839804 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3b42f3eb-8cea-4314-a6c1-2ef3afd6bee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aef73a49de304706a59335eff37debf0'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:36:16.840486 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3b42f3eb-8cea-4314-a6c1-2ef3afd6bee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aef73a49de304706a59335eff37debf0'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:36:16.847234 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3b42f3eb-8cea-4314-a6c1-2ef3afd6bee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aef73a49de304706a59335eff37debf0'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:36:16.859327 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1481/2956] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Sep 7 08:36:16 2022] DELETE /identity/v3/users/aef73a49de304706a59335eff37debf0 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:36:17.030611 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adca82-82b0-4b68-bf88-32aee8eca5d5 service placement] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:36:17.041967 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adca82-82b0-4b68-bf88-32aee8eca5d5 service placement] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:36:17.101645 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27daecc9-4d90-44d1-a078-7c9d3abef2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984123d0eaef4929af01f39a46ebd51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb18598c40b6447faef02991e1b9c4ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:36:17.102273 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27daecc9-4d90-44d1-a078-7c9d3abef2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:36:17.102551 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27daecc9-4d90-44d1-a078-7c9d3abef2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:36:17.102824 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27daecc9-4d90-44d1-a078-7c9d3abef2fb admin admin] PATH_INFO: `/v3/projects/0f785182c7d64af5ab5d37a4b95275b9` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:36:17.104639 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27daecc9-4d90-44d1-a078-7c9d3abef2fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f785182c7d64af5ab5d37a4b95275b9)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:36:17.106524 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27daecc9-4d90-44d1-a078-7c9d3abef2fb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:36:17.151443 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-27daecc9-4d90-44d1-a078-7c9d3abef2fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f785182c7d64af5ab5d37a4b95275b9', 'request_id': 'req-27daecc9-4d90-44d1-a078-7c9d3abef2fb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:36:17.156922 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-27daecc9-4d90-44d1-a078-7c9d3abef2fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f785182c7d64af5ab5d37a4b95275b9', 'request_id': 'req-27daecc9-4d90-44d1-a078-7c9d3abef2fb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:36:17.162882 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-27daecc9-4d90-44d1-a078-7c9d3abef2fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f785182c7d64af5ab5d37a4b95275b9', 'request_id': 'req-27daecc9-4d90-44d1-a078-7c9d3abef2fb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:36:17.175271 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-27daecc9-4d90-44d1-a078-7c9d3abef2fb admin admin] The token cache is being invalidate because project 0f785182c7d64af5ab5d37a4b95275b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 08:36:17.175583 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-27daecc9-4d90-44d1-a078-7c9d3abef2fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:36:17.176218 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-27daecc9-4d90-44d1-a078-7c9d3abef2fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 08:36:17.177750 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1476/2957] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Sep 7 08:36:17 2022] DELETE /identity/v3/projects/0f785182c7d64af5ab5d37a4b95275b9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 08:36:26.855946 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=5,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:36:27.172049 dv-cinder-836027-10-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 08:40:04.404359 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42f3eb-8cea-4314-a6c1-2ef3afd6bee9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 08:40:04.405806 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3b42f3eb-8cea-4314-a6c1-2ef3afd6bee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:40:04.489403 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42f3eb-8cea-4314-a6c1-2ef3afd6bee9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 08:40:04.546404 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b39096-863e-4000-bcec-0e47cbb51c0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16b81ab30294bc4a8c87b6d7c3573f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a949b959e584474aafb4a05ebbabcc4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 08:40:04.547256 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b39096-863e-4000-bcec-0e47cbb51c0d service placement] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 08:40:04.547528 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b39096-863e-4000-bcec-0e47cbb51c0d service placement] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 08:40:04.547797 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b39096-863e-4000-bcec-0e47cbb51c0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 08:40:04.549510 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f9b39096-863e-4000-bcec-0e47cbb51c0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 08:40:04.560639 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b39096-863e-4000-bcec-0e47cbb51c0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 08:40:04.562195 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b39096-863e-4000-bcec-0e47cbb51c0d service placement] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 08:40:04.572580 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1482/2958] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 7 08:40:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 08:40:14.568363 dv-cinder-836027-10-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=65 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}